www.auctionsieve.com
The AuctionSieve forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Fri Jan 01, 2021 2:11 pm    Post subject: Summary of current AS issues (and workarounds) Reply with quote

I thought it might be useful to summarise the current issues with AS - and possible workarounds.

Arrow Occasionally AS fails to retrieve the item number for an item.
(I believe this is causing this issue and this issue.)

Workaround described here.


Arrow Multiple massive errors in AS if you open all your sieves at once (400 error, 500 error, ebay is returning pages I cannot interpret, etc).
(This could be the same thing as this issue but rbryson doesn't mention lots of error messages.)
=> I believe Ebay have implemented DDOS projection. I find if I open more than 3 sieves at once (or less sieves if I'm also view [load] lots of item tabs at the same time in my browser) then AS returns many, many error messages, stacked on top of one another and it's hard to click through them all, there are so many - and I get no sieve results whatsoever.

Workaround - only open 2 or 3 sieves at once. If you've triggered this issue then you'll need to wait a while (15 minutes or so) before trying again.


Arrow Sometimes often, sometimes occasionally AS does not return the Time left on an item.
(Likely to be related to [the cause of] this issue?)
=> Refresh does nothing, it's as if ebay are hiding the item end time. These items seem to be those on perpertual renewal, i.e. those that have no actual end date (those that would go round and round every 30 days before).

Workaround - It's not fatal, it just means the item appears at the top of your watchlist (if you search by Time left). I just assume these items are ending soon, and view them quickly.


Arrow Occasionally the AS watchlist wrongly reports that an item has ended.
=> May be related to the issue below, but is most likely due to the perpetually for sale listings (going round every 30 days). Can also be when sellers run out of stock, and then edit the listing with more stock (they don't like buyers knowing they have 200 of the item - making it seem like they have only 3 left makes buyers more keen to grab whatever it is).

Workaround - check items that AS has ended as if they were normal items (it's not a fatal problem).


Arrow Often AS misses that a BIN item has been sold (has ended). Sometimes I am suspicious that AS swaps the results of ended items with details of a new item.
=> I think the way ebay handles ended items has changed & is continuing to evolve, and AS isn't picking up what has happened. Ebay seem to suggest a new item (which is of course way more expensive), rather than directing you to the old "this item has ended" page (and letting you view it again if relisted).

Examples:
https://www.ebay.co.uk/c/115842355#oid124505680481
Bizarre new item page format, which IMHO is broken. The ebay item ID is nowhere to be found - not even in the URL - and list functionality is missing from the page. Thankfully the link to the original ended item still works. "Good old ebay" making changes for "improvements" Evil or Very Mad
https://www.ebay.co.uk/c/115842355
(Same page, stripped of the "The listing you're looking for has ended." header)
https://www.ebay.co.uk/itm/373395456649
(The actual item at it's traditional URL with item number, found by searching ebay for the full title "New In Box Intel Pentium 4 640 3.2GHz (SL7Z8) Processor")

Workarounds - View items in your AS watchlists soon after you find them (don't use your AS watchlist as your main watchlist). Use ebay's lists functionality (create a "Bought CDs 01" etc list to easily discount the things you've seen before that keep going round and round). Search ebay directly (in your browser) for the title of the relevant auction when you suspect an issue.



Arrow Multiple irrelevant results in your sieves. Significantly less "good" results than you might normally expect.
=> Ebay seem to be making the results for your searches "less related" to what you are searching for - i.e. more AI, grey, fuzzily related to what you input. One of the favourites seems to be a link near the top of the results saying "We removed some search results to show you the most relevant listings. View all results" which hides a lot of the rubbish (good) but also some of the good stuff (bad). [You only seem to get this if you sort the results by any method but "Best match".] For example, search for a USB 3 cable and ebay will not show you all the USB 3 cables that are available...

=> Also ebay seem to be going overboard on "Sponsored" items (which are usually rubbish or way more expensive than what you're looking for). I think AS is returning these sponsored items as good results.

Workarounds - get good with your trash words. (This is one of the major benefits of AS.) Perform multiple narrow searches, don't just rely on one wide one to catch everything.



Arrow AS only finds a relevant item that you would have bid on with only minutes (or seconds) to spare before it ends.
=> This happens despite you having run your sieves earlier in the same day, maybe even only a few hours before. It's probably related to ebay's changes to search (making them less related) above. But also possibly the seller has multiple similar items for sale (ebay will hide the second, third etc items to "clean up" the search results) - or the seller is of low standing (poor feedback etc) and ebay are demoting their results in the searches, sometimes hiding them completely.

Workaround - run your sieves multiple times a day (but this only partially fixes the issue).


Arrow New today not working
(reported by other users here and here)

Workaround - don't use New today. Search ebay for everything that matches your search, and instead tick the "Hide items shown before" box. (This is IMHO one of the strongest features of AS.) As above, combine this with multiple narrow searches directed into the same sieve (especially if each search returns many thousands of results) e.g. search for the same thing repeatedly but at different price bands (less than $10, $10-15, $15-30 etc).

Yes it's not perfect (AS appears to have limits on how many items it will save as "shown before" both in a maximum number and a time limit) but combine this with ebay's list functionality (create your own lists of "Seen items 03") and discarding these previously seen items when they come round again is quick enough.


If I've missed something, please add it below...
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Mon Mar 08, 2021 8:25 am    Post subject: Reply with quote

Thank you Nev for finding time to look at these!
Back to top
View user's profile Send private message Visit poster's website
nev
Site Admin


Joined: 15 Sep 2004
Posts: 1110
Location: Sydney, Australia

PostPosted: Mon Mar 08, 2021 10:50 am    Post subject: Reply with quote

Thank you for the comprehensive post. It's fantastic for me to use as a guide!
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Sun Mar 14, 2021 9:03 am    Post subject: Re: Summary of current AS issues (and workarounds) Reply with quote

Thanks Nev for 3.0.6.2!

The changes made in 3.0.6.1 certainly seem to have dramatically reduced the number of results I'm getting - I don't seem to be seeing the items going round every 30 days that I used to.

(EDIT - to be clear, the results below are after testing 3.0.6.2)

wickmesh wrote:
Arrow Occasionally AS fails to retrieve the item number for an item.

Unfortunately this one is not fixed, sorry! (Grey highlight on example item below is due to mouse cursor position)



If I click on this item, it leads to the URL https://www.ebay.co.uk/itm/ (missing the item number) and I get the "We've looked everywhere" page. This particular item I know I've seen before (correctly in a sieve result) and if AS had picked up it's item number, it would have been excluded from my sieve results this time.


wickmesh wrote:
Arrow Multiple massive errors in AS if you open all your sieves at once (400 error, 500 error, ebay is returning pages I cannot interpret, etc).

I've not seen this for a while - but I'm usually being careful and opening only 2 or 3 sieves at once.

Deliberately provoking this one (opening every sieve I can at once) didn't bring up the whack-a-mole errors - but previously it did seem to vary depending on the time of day and hence the load on ebay's servers (before 9am on Sunday when I tried this, they're unlikely to be that busy).

(Aside and nothing to do with AS - I think ebay's DDOS protection has also affected their list functionality, I'm getting rather fed up with having to repeatedly refresh an item to get my lists to show up in the item listing, so I can add them to the list I want! Doing so is of course hammering their servers... <rant>Ebay's list functionality is at best clunky, at worst broken!</rant>)

wickmesh wrote:
Arrow Sometimes often, sometimes occasionally AS does not return the Time left on an item.

Still a present issue. (I've unticked Hide items shown before to get these results). The examples below are shown in the sieve and then in the watchlist.



5 items in the image above:
https://www.ebay.co.uk/itm/174389052165
https://www.ebay.co.uk/itm/274632627907
https://www.ebay.co.uk/itm/174320891906
https://www.ebay.co.uk/itm/184581182099
https://www.ebay.co.uk/itm/174616124378

All the above items are BIN and are probably on perpetual renewal (going round every 30 days). Refresh (once in the AS watchlist) doesn't get then end time data. None of these items above show an end time (Time left) on their item page - but then again, nor do these 3 example items below, but these all have ending time data in AS:
https://www.ebay.co.uk/itm/203104633463
https://www.ebay.co.uk/itm/202782544770
https://www.ebay.co.uk/itm/183952072655

wickmesh wrote:
Arrow Occasionally the AS watchlist wrongly reports that an item has ended.
Arrow Often AS misses that a BIN item has been sold (has ended). Sometimes I am suspicious that AS swaps the results of ended items with details of a new item.

Not seen these ones since your updates, but then again I've been following my own advice and rarely using the AS watchlist.

wickmesh wrote:
Arrow Multiple irrelevant results in your sieves. Significantly less "good" results than you might normally expect.

Early days - but as above I'm seeing less results, full stop. But I do seem to be seeing less "rubbish" also.

(Looking at the results from old sieves [now-retired sieves for things I've bought] when trying to provoke the whack-a-mole error - there does seem to be rather less rubbish now.)

wickmesh wrote:
Arrow AS only finds a relevant item that you would have bid on with only minutes (or seconds) to spare before it ends.

This hasn't happened for a while. I'm thinking this was not an AS problem. But then again, I'm running my sieves more often than usual.

wickmesh wrote:
Arrow New today not working

Others will have to say if this is fixed, it's not a feature I use.
Back to top
View user's profile Send private message Visit poster's website
nev
Site Admin


Joined: 15 Sep 2004
Posts: 1110
Location: Sydney, Australia

PostPosted: Sun Mar 14, 2021 9:59 am    Post subject: Reply with quote

Ok, new beta 3.0.6.2 is out.
I'm fairly confident I've covered pretty much all the issues.
Let me know if there's anything remaining.

In summary....

Blank item numbers: were sometimes happening for the last item on a result page due to other guff that eBay was adding.

Lots of errors running lots of sieves: I haven't done anything specifically for this. AuctionSieve already has some throttling to make sure only 10 requests are running at once.

Time left: yes, unfortunately ebay now doesn't return the time left for a lot of buy it now items - nothing I can do about that but I have improved the time left scraping

Wrongly reporting item has ended: pretty sure I've fixed this now - doing better identification of ended vs non-ended items

BIN sold items: the item page for an ended item has sponsored items above the ended item details which was tripping up AS. This is now fixed so it should detect better whether an item has ended

Multiple irrelevant results in your sieves: I've added new detection for the extra stuff ebay keeps adding - eg "items from international sellers" and "items matching fewer words"

AS only finds a relevant item that you would have bid on with only minutes (or seconds) to spare before it ends: This one can still occur - it will happen on searches where there's an item that ends while AS is still downloading the results. Eg page 1 downloads with 50 items, then first item ends, then page 2 downloads and what would have been item 51 is now bumped up to item 50 on page 1, so AS never sees it. Can't do much about this really.

New today not working: hopefully this is fixed
Back to top
View user's profile Send private message Visit poster's website
nev
Site Admin


Joined: 15 Sep 2004
Posts: 1110
Location: Sydney, Australia

PostPosted: Sun Mar 14, 2021 10:04 am    Post subject: Reply with quote

Ha, I didn't see your post before posting my post.

So, the one remaining issue that can be fixed is this blank item number thing.
Can you give me the details of that search?
Back to top
View user's profile Send private message Visit poster's website
nev
Site Admin


Joined: 15 Sep 2004
Posts: 1110
Location: Sydney, Australia

PostPosted: Sun Mar 14, 2021 10:38 am    Post subject: Reply with quote

Ok, fixed it.

I just did a search for exact phrase "vintage gold ceramic ic" and got 2 results including that dodgy one. Did some testing and figured out a better way of getting the item number that works.

Let me know how you go!
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Sun Mar 14, 2021 10:50 am    Post subject: Reply with quote

nev wrote:
Lots of errors running lots of sieves: I haven't done anything specifically for this. AuctionSieve already has some throttling to make sure only 10 requests are running at once.

Exclamation An idea (that would help in other whack-a-mole situations too): Can another option button be added to all of the error boxes "Stop all sieves"? This could stop all active sieves without having to cancel all the active errors and click on each tab before more errors pop up - stopping the whack-a-mole game much faster.

EDIT - the button has to be on the error boxes, as these are brought to the front and you can't click anywhere else until clearing them (of course).

nev wrote:
Time left: yes, unfortunately ebay now doesn't return the time left for a lot of buy it now items - nothing I can do about that but I have improved the time left scraping

Understood, thank you.

nev wrote:
AS only finds a relevant item that you would have bid on with only minutes (or seconds) to spare before it ends: This one can still occur - it will happen on searches where there's an item that ends while AS is still downloading the results. Eg page 1 downloads with 50 items, then first item ends, then page 2 downloads and what would have been item 51 is now bumped up to item 50 on page 1, so AS never sees it. Can't do much about this really.

Understood, makes perfect sense once explained. You can have exactly the same issue when searching the site manually. So the fix for this is as I'd found - to run my sieves more often (at least every 12 hours).

(The other option would be to have AS automatically run through all the searches in a particular sieve twice - but of course this doubles the search load on ebay and may invoke the DDOS protection problem - for only a very few extra results.)

nev wrote:
So, the one remaining issue that can be fixed is this blank item number thing.
Can you give me the details of that search?

Of course. The issue is not consistent, but the exact same example item above (https://www.ebay.co.uk/itm/254833220887) keeps appearing without its item number (3 times so far today). I think it's this search: "scrap (cpu, cpus, processor, processors, intel, amd)"

EDIT - I wrote the above before your reply, looks like you've found the issue by other means. Thank you! Downloading 3.0.6.3... will test for a couple of days and let you know.
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Sun Mar 14, 2021 10:57 am    Post subject: Reply with quote

Sorry, I don't need a couple of days! That damn same item is still there...

Back to top
View user's profile Send private message Visit poster's website
nev
Site Admin


Joined: 15 Sep 2004
Posts: 1110
Location: Sydney, Australia

PostPosted: Mon Mar 15, 2021 8:26 am    Post subject: Reply with quote

Hmmm, I can't replicate this.
I managed to replicate the problem before and made a change which solved it for my search.
I think I'll need even more details.

For example, if I have this:
Search term: scrap (cpu, cpus, processor, processors, intel, amd)
Use: ebay uk
All of these words
Time frame: All
and nothing else checked

I get 30 items including that 4004 and they're all fine.

Can you come up with some search details that cause it reliably?
If it's random - sometimes occurs, sometimes not, I'll give you some other instructions.
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Sat Mar 20, 2021 11:26 am    Post subject: Reply with quote

Thanks for persevering on this one Nev!

For a few days this one was appearing reliably on 3.0.6.3 (same item each time), but now it doesn't appear (although the item is still live). Previously (before 3.0.6.3) the issue was pretty random (different items at different times) and although fairly common, almost unrepeatable (not the same item each time).

I'm certain it is this search (below) from the sieve that was causing it (as none of the other searches in the sieve have search terms that could find this item). But historically (before 3.0.6.3) I have found the issue on many different sieves.

Time frame all
ebay UK
scrap (cpu, cpus, processor, processors, intel, amd)
Include store items
Max price 55
Trash Word filter
Hide items shown before

I've tried a test see if I can find other search terms that cause it reliably, but no luck yet. What else would you like me to try please?

--------------------------------------------

Also I'm having (have had) an issue coming from the same item on 3.0.6.3 - I copied it into my watchlist to see what would happen. AS decided it had ended, as obviously no item to find. Now every time I open AS I get an error message (copied below). I click OK and it's gone, and there are no items in my watchlist in AS.
Code:
AuctionSieve version 3.0.6.3
Operating System = windows 10
Java version = 1.8.0_201
Java vm version = 25.201-b09
Java runtime version = 1.8.0_201-b09
Current dir = C:\Users\wickm\Dropbox\AM Docs\AuctionSieve
User home dir = C:\Users\wickm
Tmp dir = C:\Users\wickm\AppData\Local\Temp\
Tmp dir being used = C:\Users\wickm\AppData\Roaming\AuctionSieve\tmp
Problem line <      OQ 1xx Vintage GOLD ceramic IC for collection or scrap like Intel C4004 :)      £4.32   -   0   true   false         uk   1615794518529   true   >
hashbang.io.BadLineException: Item id is blank
   at hashbang.auctionsieve.itemdb.EbayItemDB.initialise(Unknown Source)
   at hashbang.auctionsieve.itemdb.EbayItemDB.<init>(Unknown Source)
   at hashbang.auctionsieve.itemdb.PriceHistoryEbayItemDB.<init>(Unknown Source)
   at hashbang.auctionsieve.itemdb.PriceHistoryEbayItemDB.<clinit>(Unknown Source)
   at hashbang.auctionsieve.itemdb.PriceHistory.<init>(Unknown Source)
   at hashbang.auctionsieve.itemdb.PriceHistory.<clinit>(Unknown Source)
   at hashbang.auctionsieve.itemdb.WatchListPanel$1.run(Unknown Source)
   at java.lang.Thread.run(Unknown Source)



Entire contents of pricehistory.txt
Code:
VERSION3.0.6.3
      OQ 1xx Vintage GOLD ceramic IC for collection or scrap like Intel C4004 :)      £4.32   -   0   true   false         uk   1615794518529   true   


The fix (for me) was simply to delete the item line from pricehistory.txt and the error on opening goes away - but maybe AS should catch/deal with this to stop it recurring?
Back to top
View user's profile Send private message Visit poster's website
nev
Site Admin


Joined: 15 Sep 2004
Posts: 1110
Location: Sydney, Australia

PostPosted: Sun Mar 21, 2021 11:00 am    Post subject: Reply with quote

Ok, still can't replicate with that search.

However, new beta released with a fix for that bad line exception - basically if an item doesn't have an id, I won't save it to the watchlist any more.

Also a feature you requested a while ago!
Go to Tools->Options...->Sieve actions and turn on "Show search number column"
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Tue Apr 13, 2021 6:45 am    Post subject: Reply with quote

Just had this (no item number) happen again - but this is the first time in a good while. Seems (for the moment) to be consistent...

Same item! https://www.ebay.co.uk/itm/254833220887

Same search as before (which is easy to confirm now thanks to the search number column Very Happy )

Also - nothing stops me adding it to the watchlist - but if I restart AS it's gone from my watchlist, not in my price history, and there's no error message.

Running AS 3.1.0 on Win10.
Back to top
View user's profile Send private message Visit poster's website
wickmesh
AuctionSieve guru


Joined: 04 Jun 2008
Posts: 232
Location: Burton on Trent, England

PostPosted: Sat Apr 24, 2021 10:17 pm    Post subject: Reply with quote

Starting to get this "no item number" problem in other searches/sieves now.

Result "floor mat sound insulation board thickening treadmill mat shock absorption~" from search

Time frame all
Use ebay.co.uk
Search term
("golds gym", proform, "nordic track", NordicTrack, healthrider, treadmill) (board, controller) -silicone -running -covers -yijian -cable -harness -display
All of these words
Include store items
Max Price 200

Searching ebay directly finds this item https://www.ebay.co.uk/itm/164767727304


Result "BOS Best of Show Land Rover Range Rover SUV SUV available 1:87 H0" from search

Time frame all
Use ebay.co.uk
Search term
bos (landrover, "land-rover", rangerover, "range-rover", defender, "land rover", freelander, discovery) -1/18 -1/43 -wiper -bosal -bosch -brake -fuel -pillar -capping -seal -belt -hose -glow -net -"boot tidy" -solenoid -bearing -trim -throttle -paperback -opener -steering -shelf
All of these words
Sellers to exclude boxter7248 shirley9293
Include store items
Show items from preferred locations Worldwide
Max Price 100

Searching ebay directly finds this item https://www.ebay.co.uk/itm/254683317930

Currently these two are consistently received every time I run the sieve concerned.

The last character(s) of the item titles of 2 of 3 of these results is strange (not a standard character), could this be what is throwing AS off?
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group