Page 3 of 4

Re: RootsVinylGuide.com

Posted: Sun Oct 25, 2009 9:35 pm
by Jah Chicken
@credezcrew24: Oh really? Thanks for the info. I'm curious about the price of some of these records because there are so much vinyl showing up that I know nothing about.

Site Updates (for http://rootsvinylguide.com ):
Added tracking of UK ('ebay.co.uk') site as a portal in order retrieve results that are filtered from the US ('ebay.com') site. It'll be interesting to see how many more auctions are tracked.

Auction result currency is now displayed in the currency native to the sellers country.

Roots,
Jah Chicken

Re: RootsVinylGuide.com

Posted: Mon Oct 26, 2009 2:42 am
by credezcrew24
Really you never knew about "Treat Me Good" Its really Peter Tosh and The Wailers but miscredited to The Beltones on the original release. Its always been one of my favorite tunes.

I love your site and it will definitly help alot especcially in the future when it gets more auctions listed. I was wondering how exactly does you site take the data from ebay and put it on your site?

Re: RootsVinylGuide.com

Posted: Wed Oct 28, 2009 4:47 am
by Jah Chicken
Greetings,

Huh, I didn't know that one but I definitely want to hear it! Especially if it's Peter Tosh on vocals with The Wailers. Thanks for the information.

As for fetching the data from EBay, I make use of their developer apis ( http://developer.ebay.com ). I made my own EBay client that does the following:

* Searches EBay sites for auctions ending soon in a specified category (roughly Records -> 'Reggae & Ska').
* Track the EBay item id and auction end time
* After the auction has ended, fetch the data for that item and add it to my application if it has been bidded upon.

This process is repeated every 20 minutes for the time being. My web application handles serving up all the data (retrieved by the Ebay client) in human readable format.

I hope this helps explain a little bit.

Roots,
Jah Chicken

Re: RootsVinylGuide.com

Posted: Tue Nov 24, 2009 9:04 pm
by Jah Chicken
Greetings,

Just a quick update:
* As of a few weeks ago, auctions are now browsable by record duration (LP, EP, and single). It is common that a seller mis-categorizes their record or uses a non-standard tag for duration. Those records will appear in the wrong duration category or in the 'other' category respectively.
* Currently the site has tracked a total of 20292 auctions.

Roots,
Jah Chicken

Re: RootsVinylGuide.com

Posted: Tue Nov 24, 2009 10:49 pm
by Rootikalist
amazing work , in few years it will be much better than popsike!

Re: RootsVinylGuide.com

Posted: Thu Dec 10, 2009 9:53 pm
by Jah Chicken
@Rootikalist: Many thanks brethren!

Greetings,
I seek your valuable input for the next phases of development of rootsvinylguide.com. My next development phase will primarily consist of standardization techniques of the growing data catalog on the site. Some issues that I am facing right now (some of which I have mentioned previously) are as follows:

* Sellers often miscategorize item specifics (record size, duration, etc).
* Sellers title auctions in non-standardized ways (i.e. "PROPHET/CONQUERING LION - VIVIAN JACKSON [ROOTS REGGAE]" vs. "VIVIAN JACKSON - CONQUERING LION - PROPHET").
* Sellers don't always know important data like the title or even the artist of a song.
* EBay does not offer item specific fields for valuable information like matrix # and record label (*shakes fist*).

Through observing the data over time, I believe that many of the title are similar enough to do some programmatic comparisons and ultimately standardize the listings. Of course, some titles vary so much that this will not be an option. In order to achieve any computational standardization I will need a canonical index of (read: discography) of all reggae vinyl. This would be the 'pie in the sky' so to speak, but is an unrealistic goal at this time. The way I would go about starting to build this index is to crawl sites like Tapirs, and maybe even talk to the Roots Archives moderators (hey guys!) about extracting some of their data. However, writing crawlers for sites that are designed to externalize their data in a computer friendly way is dirty, dirty business.

Another way to standardize the data is to make it easy for site users to generate new standard records and map auctions to them. That way, this 'discography' would be all user contributed. I believe that this kind of feature uptake would totally depend on its ease of use. For instance, if the site always would try to suggest a new standard mapping for an auction or would attempt to connect it to an existing mapping that it has a good guess about and a user could 'sign off on it', then I think it would reduce the amount of arduous work load that is the core of my fear with this approach.

These last 2 ideas could exist in harmony. I could do some crawling in additional to building the community approval type process. I think that there is a good balance to strike there.

Also, one more thing involving currency. I changed all currency from USD to the currency native to the sellers country and am not quite satisfied with it. I think the most appropriate thing to do is assume that whoever is using the site would like to see the prices displayed in the own native currency. I originally thought that it would be slightly more accurate to always display the sellers currency but I now believe that a consistent currency viewing experience is ultimately more desirable. Any thoughts on this?

So, that's about where I am at philosophically at the current time. I would love to hear suggestions, feedback, or other ideas of ways to standardize the data! Many thanks.

Many Thanks!
Jah Chicken

Re: RootsVinylGuide.com

Posted: Sun Dec 13, 2009 2:41 pm
by kalcidis
Jah Chicken, your site is GREAT! The only problem I have with it at the moment is the rather dull design. But I'm guessing this will be sorted out in the future. I check roots vinyl guide even more than popsike nowadays.

Re: RootsVinylGuide.com

Posted: Mon Dec 14, 2009 12:22 am
by ruff
Nice idea, about time someone did something like this, good on ya!
Many times i think to myself how the ebay sales data needs to be catalogued, i just hope it is kept on your own servers and not on ebays so u have a backup.
I really think your site can do a lot to help to contribute to projects like RKR, if it's still going that is, or other similar reggae vinyl cataloging projects. I see so many obscure titles on ebay, will be nice to have them recorded.

great job....

Next thing is to publish a book of your results and also include some graph displays for record values over time period etc., i've not checked your site yet but hope it lissts min+max selling price and number of items sold out of XX number of auctions like ebay lists thier own advanced sales reports. This will help sellers guage how well records sell and the chance they have of selling it etc.
Gonna take a look now....

Re: RootsVinylGuide.com

Posted: Thu Dec 17, 2009 8:18 pm
by Jah Chicken
@kalcidis: Many Thanks! I absolutely agree about the design. I think that may be the next piece of relatively low hanging fruit that I want to take on. I'm not a very good designer, but I certainly have ideas about making the site more usable. Somewhere down the road I'd love to work with a graphic designer to help make the site look really nice.

@ruff: Thanks for the feedback! Price trending for items over time is my goal. The main problem I'm facing is how to identify if two auctions are the same. I have listed some ideas about this in my previous post. However, I think that graphs for existing data could still be valuable. For instance, a graph of all record sales for a day, or a graph for the price of all search results. I'm not sure whether or not I'll start graphing data before the standardization initiative I want to start. Also, I'd love to contribute this data back to the Roots Archives if they except it. Long live RA!

Unfortunately I haven't been able to work on the site much this month because of external factors: moving apartments, holidaze, etc. but I'm really looking forward to stepping it up a notch in the next coming year. Thanks again for the feedback!

Roots,
Jah Chicken

Re: RootsVinylGuide.com

Posted: Fri Mar 19, 2010 2:04 am
by Jah Chicken
Greetings!

To note, I have added a few visible updates:

* You many now sort any search query or category browsing (LPs, Singles, etc.) by end time, pice, or title. The sort may be in ascending or descending order. Descending is the default but you can simply click the same sort link again to toggle the order. The interface is fairly poor, but I'm planning somewhat of a redesign so I didn't want to spend too much time on it. Any graphics designers want to get involved??

* Images are now all hosted locally on the site. For a while, I was linking to the images on EBay. It looks like EBay keeps the images around for 3 months and then removes them, resulting in a whole bunch of broken image links. The images for the first 20,000 or so auctions have been lost (*sigh*), but going forward all the images will remain viewable (*yay*).

One sort that you may find interesting is browsing through all record auction in descending price order. here's the link: http://www.rootsvinylguide.com/all?sort ... order=desc. There's some pricey ones on there!

I hope that people continue to find the site useful and please let me know if you have any more great ideas.

At the time of this post, the site has track around 67,000 auctions.

Praises,
Jah Chicken