Which brings up an interesting observation, a web browser is not a web browser anymore if it is generating its own unique http post requests that are not otherwise available in the accessible http source when accessing the site. At which point you are not passively browsing, you are actively generating server requests.

Such was the case with our Russian fare friends with the "TARIFF" modification in the post string that does not exist anywhere on their website.

But to answer your question, the browser http header can be filled with whatever you want. Could be nothing or could be "User-Agent: KVS 7.5 (Scrapping your sh!t)". The sites have no idea about KVS because the web requests are coming from your computer (IP) and with your unique session information (cookies) at times with your credentials (login id, password). But you do hit on a good point, automatic agents such as this are supposed to follow the robot.txt rules.

I'm guessing if someone was able to put together the evidence for flight stats that the heavy traffic without ad image opening was due to solely to KVS Tool and not a widespread user group (as what it would appear from their server end with all the IPs) then maybe they would reconsider their business decision.