Excessive downloading - possible robotic

[ozziemate]

Joomla 2.5.8
all extensions up to date.
Phoca download.
Custom templates.
========================
Hi,
I have installed this great product on a couple of development sites and a couple of client sites.
I was amazed to find that on non-SE- indexed development sites with zero normal traffic, someone/thing was downloading files at what appeared to be a random rate of each file 2 or 3 times per "run". This "attack" occurs with a weekly frequency.
What is even more amazing is this is happening whilst these development sites are protected under a DNS routing cloud based system that is supposed to stop spam bots/and other robots from doing this type of bandwidth theft.
Another interesting thing is that even when making the categories accessible to "special" users only [ leaving the files public ], the files are still being excessively downloaded.

The only way I have found to prevent this type of downloading is to make all files available only by registered or above access levels.
I have had to think about this issue as there appears to be no motivation that makes sense. The files being downloaded are charity newsletters and have no intrinsic value to any one other than information about the charities.
This leads me to believe that the software is being deliberately targeted for harassment purposes, attempting to force the software into uselessness, due to persistent and wasteful bandwidth usage. [either that or the development sites generally are being targeted ]
Another key indicator is that even after banning numerous Ip addresses from accessing the site the downloader would re-appear using new Ip addresses [ this clearly indicates a robotic downloader, as the random downloading seems to stay with in a "range" of deviation ]

Given that the domains and there web sites are DNS routed through a security system and knowing that Phoca products are being plagued by significant spamming attacks [ ie. guest book etc] leads to the belief that Phoca products are being "specifically" targeted in a rather malicious manner possibly in an attempt to compromise Phoca as a commercial entity.
======================
There is a possibility to discover whether Phoca products are being deliberately targeted.
I am prepared to install a dummy Phoca download package onto an existing live site under the DNS routing security system and attempt to fathom how the robots are getting through the security systems. [ as all other highly aggressive Chinese and Russian Spamming bots are refused access] it is rather intriguing and somewhat disturbing to see this sort of spamming/downloading still occurring.

And I happen to like Phoca products and wanted to install the guest book but can not do so until this issue sees some progress.
Note:
Jan, you may wish to move this topic somewhere else on the board.

[Jan]

Hi, all sites with all contents (mostly forums) are visited by aggressive robots - this is why you can see everywhere in the world sites which have protection against robots (some sites e.g. run after 5 seconds you have visited them as they check your access) ... The software itself does not have any chance to solve it, so this needs to be protected on server side (checking it e.g. with apache - e.g. mod security) and in the layers above them, etc. ... I by myself have the most problems with aggressive robots here in this forum (so the server mostly down )

[ozziemate]

I just get the impression when comparing the attacks on other non -Phoca extensions I have installed including kunena etc that your software may be specially targeted. Especially as the downloading of files that are useless on developments on a couple of sites seems to crop up as soon as I install the download software. [yet none of the other non-phoca extensions appear to be effected.]

Maybe I am over sensitive but I have to seriously consider whether or not to give the extension a miss or adulterate the script to allow only "x" number downloads per IP per session...
Also have you considered cloud based DNS routing as a possible way of protecting your sites?
Currently I have a couple of kunena sites with registrations not being spammed... due to this cloud based security system. [averaging about 20 to 30 attacks per day but are stopped by the routing system.
Have you had similar reports from other users of Phoca products?

[Jan]

Hi, I am planning to do limits for Phoca Download but from my experiences this does not solve any problem because the problem is the access on the server.

If there is some belligerent robot, then it does not matter if it gets page with download or page with information it is not able to download. You allways needs to load e.g. all joomla framework, by every attempt. This is the problem, stopping belligerent robot in Joomla! script does not have any sense as you get your site overload in every case as displaying the phoca download page or displaying the page with information about not possible download takes in fact the same memory demands (I am now speaking e.g. about downloading small files which can be large as the producing html code with joomla)...

So the protection should be done not in the script but one layer or more over the php script (e.g. in apache, etc.) With apache you should stop the belligerent robot and you should save a lot of sources as you don't need to run the scripts.

So yes, I will do such feature (stop downloading for some IPs) but this will not solve the main problem, overloading the server with attempts -this cannot be solved per php script

Regarding my sites:

Also have you considered cloud based DNS routing as a possible way of protecting your sites?

Not able if I can get rights to do such feature, but thank you for the info, I will take a look at this possibility.

Jan