The first thing I need is pretty obvious. I need a list of the available items. At least by name but ideally with as many properties as I can gather. Some of the properties I know of are: item level, type, and rarity.
I know that the website has some documentation on API's that are available but there wasn't one for this list. There are pages that have the data though (Item Data). My plan is to get a list of the links there and download those pages. Then create a process to scrape the data. Technically I could do it manually but that would take a while and this is much simpler.
So really my first task is going to be creating an import of the scraped data.
I am creating a console app that can pull back the HTML and scrape it for the data I need. It is slow going but I think will work out fine.
The repository is on my github at POELootFilter.ItemScraper. It's not complete but as I get things done the updates should start to appear.
I have the program parsing the html that I am able to grab. I am up to the point of needing a database to store that info. I am unfamiliar with anything but SQL so I am researching alternatives. I also emailed GGG because some of the links were giving 404 messages when they previously worked so my item list is not complete.
The project is on hold for a little while until I figure out if I should continue or not. There may be big changes coming for POE relatively soon that would make all this effort more or less useless. I have decided to wait to see what their news is in the next couple months.
There are two other sites that have similar features though that I have found in the meantime.
https://bschug.github.io/poedit/poedit.html and http://filterblast.oversoul.xyz/
I didn't look at them too in-depth but they appear to at least give the visual cues well.
Back when I was doing all this that wasn't around. Totally solves the problem I was having.
Related Posts:
Previous:
Starting Out