2

I have network traffic details, which consist of client IPs accessing a web server. Alongwith that I have the session ids through which the client IPs access the web server. In the session details I have the objects which the client IPs access through the sessions.(objects combine to make a web page,for example there are 10 objects in a web page).

I want to find which are user generated requests and which are bot generated based on the manner in which objects are requested.

rohit
  • 21
  • 2

1 Answers1

2

You have no guarantee that you can detect bot-net activity from object request behaviour, because ultimately it comes down to software - a user uses a web browser and a bot-net zombie can emulate this and a typical user browsing pattern if it desires.

Typically zombies will not be able to execute JavaScript and so may choose not to retrieve static scripts or even styles but then it depends what the zombie is doing. However if it wants to cause DoS then it may retrieve every resource on the page it can simply to generate more traffic.

So rather than looking at the specific objects requested, try building a pattern of the objects requested. Is the user in the same session following a typical browse pattern, e.g. their first request is of the home page, then the forums, read an article, etc.. The issue with this approach, if you are suffering from bot-net visitors, is your server may already be overwhelmed by the time you have enough data to start null-routing zombies.

deed02392
  • 4,128
  • 1
  • 21
  • 20
  • yes, I am planning to build the pattern from the objects, and assuming that this way will work which algorithm might work to differentiate bot from user.As of now my aim is not to make it a run time model. – rohit Nov 27 '13 at 13:13
  • I mention some approaches in my third paragraph. I am not sure of any existing public algorithms you could adopt, but you have to consider the usage patterns would be different depending on the site content. Perhaps you could document known-good visitors and base your algorithm on identifying non-conforming patterns. – deed02392 Nov 27 '13 at 13:25
  • 1
    Imlpementing javascript in a bot is hard - but conversely it's then difficult to discriminate between a real user who has javascript disabled and a simple bot. OTOH implementing CSS in a bot is even harder - hence seeding your site with links in hidden divs pointing to traps will likely be more effective - combine this with navigation using a URI bound to and derived from the session token prevents the bot from following a pre-determined path. – symcbean Nov 27 '13 at 13:32