Most likely bots. Maybe offline readers or read ahead cachers other automated tools.

 

Baidu is an aggressive harvester, but I think it honors robots.txt. I've never seen real searches referred from there, but it's data might feed search sites that your users visit. Put your PHP redirector in the robots.txt file.

 

 

 

 

 

 

 

 

 

 

 

 

 

whatever your php redirector page is (auction.php, redirect.php, etc), add it to robots.txt - bots that respect the file won't click your redirector.  robots.txt isn't a blocker, it's a directive to bots.

 

QCT and other redirectors deal with clicks after the fact, robots.txt tries to prevent them up front. You don't need special knowledge of the bot (such as IP). That should filter out respectable bots and leave you to address the violators.

 

 

I see it clicking eBay links from tools other than PhpBay (those not tracked) several times

 

How are you seeing the clicks if they aren't tracked? EPN doesn't provide UA/referrer data.

 

When we talk about clicks here, we mean they load your page (OK) and then click/open the eBay links on the page, inflating your click count (not OK). For standard rover links, the page load appears in your webserver log, the click does not.

 

You can't block bing entirely without losing your listings on Bing search.

 

 

When I say I see it clicking links I can see them in the visitors log, the url is shown and it is the eBay strings from one of my tools.

 

Not sure what you meant by that. Web logs show page requests not clicks. If you display a link like  http://rover.ebay.com/etc on your page, and a user clicks it, the webserver does not track that unless you have click tracking setup in javascript

 

So what you are saying is that just because that is the page Bing is on, does not mean they successfully clicked that link?

 

Right. Unless you are compelling them to click somehow, or forcing the click, which are TOS violations.

 

Sequence is like this with no click tracking

- page request (logged)

- display http://rover.ebay.com/etc

- user clicks

- client sends user to eBay

(eBay tracks click)

 

With javascript click tracking

- page request (logged)

- display http://rover.ebay.com/etc

- user clicks

- javascript executes and sends tracking data to a server (same server or third party)

- client sends user to eBay

(most bots never click, they request the link directly from eBay, EPN tracks it as a cllick because they can't tell it isn't, and your click tracker never fires)

 

With redirector script

- page request (logged)

- display http://myserver/redirector.php?item=etc

- user clicks

- client sends user to local redirect script (logged)

- the redirect script checks for bot and blocks or modifies links as needed

- the redirect script logs click data to file or database

- the redirect script sends redirect to client

- client sends user to eBay

(Bots can't click straight thru to eBay on redirect links. All clicks go thru redirector, where you can block them in a variety of way, or pass them thru without roverizing. Installing a redirect script should result in a quality improvement.)

 

It's unlikely that a bot is clicking only the direct-to-ebay links and not the PHPbay links. So your bots are probably already logged in your QCT logs.

 

Not sure what level of reporting QCT provides, but if you can get click counts by IP address, you'll have a list you can use to spot the top clickers, the top level of which is your bots. If you have 4 items per page, bots will click all four. Users will click one.

 

Include the client IP in the customid. Then you can use customid to compare the click counts from QCT with the sales in the EPN transaction log. IPs that click a lot and never buy anything are bots (or chronic window shoppers). You should look back a month, not excluding very recent clicks where the eBay sales may not have closed yet. Block the IPs of the high clickers, no sales and any IPs that QCT flagged as bots. That should block those bots from clicking the direct-to-ebay links.

 

 

Is it possible that there is a bot or activity that will not record in latest visitors?

 

Yes, I mentioned that several times. This is the common bot effect.Basic web logging records page views not clicks. Outclick trackers only record events that occur on your website. Bots don't click, and they don't do thru your website unless you force them to (redirector) - they just fetch the page, extract all the links, fetch link, fetch link, etc.

 

http://www.devbistro.com/articles/Misc/Implementing-Effective-Web-Crawler  (see pseudocode)

 

http://en.wikipedia.org/wiki/Web_crawler

 

 

Here is a bot that just hit me: ezooms bot.  Information about it on the following webpage (not affiliated with me, it just summarises the info about the bot well):  http://graphicline.co.za/blogs/what-is-ezooms-bot

I block anything that has "bot" in the UA after applying a few exceptions.

 

ezooms is an anagram for seomoz btw

 

 

192.114.71.13 still top offender. subnet is now blocked permanently, no further reporting on that one. Israel, picscout

27.153.187.70 new chinese offender last three days.
related to click spike thread? - "EPN Clicks 30% Higher"
Spoofs multiple agents

Abusive bots by IP:
124.115.1.60
4.79.204.36
91.218.245.253
194.44.170.227 <-- googlebot spoof?
141.105.65.68
124.115.1.62
91.237.249.91
109.254.6.14
193.107.17.247
218.93.127.117

IPs using GoogleBot agent, but maybe spoofing
http://www.projecthoneypot.org/ip_46.232.228.188
http://www.projecthoneypot.org/ip_194.44.170.227
http://www.projecthoneypot.org/ip_72.167.131.129
http://www.projecthoneypot.org/ip_79.180.192.40
http://www.projecthoneypot.org/ip_95.211.190.89

Nothing good can come of these UAs:
OffByOne: Mozilla/4.7 (compatible; OffByOne; Windows 2000) Webster Pro V3.4

That makes sense and is how I selected the IPs I listdd for gbot, excludjng the commonly used subnets. I just started looking into the gbot spoofers and haven't looked into each range much. What's the IP range for 1e100.net?

Yeah, the transcoder is that proxy for mobile devices. But it doesn't identify as googlebot. The ones I indicated as spoofing are actually using the GoogleBot UA.

 

The 209 traffic comes in with a lot of referrers, sometimes off bing and yahoo. I have some sales from that range so there's probably employee or customer traffic on those subnets.

 

I see over 500 distinct IPs last month from those ranges, 8 of which were flagged as bots (high volume, no purchases), and 8 that bought items. Some had multiple agents, so maybe a proxy?  I'll look at that some more when I have some time.

 

I'm not blocking the bots, just excluding them from my reports.

 

 

OK more info:

 

The real googlebots I see are resolving to googlebot.com and using the UA: "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

 

The active fakes are using the same UA or just "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" and coming from RIPE

 

There are a few single clicks as well, but those are probably just developers simulating googlebot that forgot to change their UA back or something similar. (Canada, US)

 

The three main fake googlebot IPs from last month are:

194.44.170.227 Ukraine

46.232.228.188 Germany

95.211.190.89 Netherlands

 

 

 

 

 

 

 

 

 

 

 

eBay DE clicks: Nope, nothing. I never really pressed the issue actually; too much other stuff goin' on.

Googlebot spoofing: saw that Ukraine IP myself; on the blocklist.

[Some Handy Aggregated Reference Info for ePN Affiliates and eBay Developers]