How To Block Fasterfox Requests - February 15, 2006
Over the last few months the amount of visitors using Mozilla Firefox has grown to about 25%. The amount of bandwidth being used has also increased a large amount. Part of the reason behind this is that many Firefox users use an extension called “Fasterfox”. This extension “pre-fetches” links on a page so that if the user were to click on a link it would load much faster because its already been downloaded. This may be more convenient for viewer, but is a major problem for many webmasters who are low on bandwidth. Since Fasterfox constantly requests new files, it can cause many servers to overload much faster than if a person viewing the same content without Fasterfox were to view it.
Fasterfox is in fact one of the most popular extensions for Firefox. It is currently ranked as the 3rd most downloaded extension for Firefox on Mozilla Update page. (aka. Firefox Add-ons) The latest version of Fasterfox, v1.0.3, checks the for the robots.txt file on the site the viewer is visiting to check whether it should pre-fetch or not. This new feature allows webmasters to add the following text (in bold) to their robots.txt file to prevent Fasterfox from pre-fetching links. Text To Add To “robots.txt”:
- User-agent: Fasterfox
- Disallow: /
Adding these two lines somewhere in your robots.txt file and placing it in the root folder will prevent Fasterfox from pre-fetching links anywhere on your site. (ex: yourwebsite.com/robots.txt) Webmasters can modify the text so that Fasterfox will only be prevented from pre-fetching on specified directories. Still have questions? Ask us! (reply in the comments)
Links: Fasterfox | Firefox Add-ons
Related: “robots.txt” Tutorial
Recent: Firefox Extend Contest Finalists
posted by Sahas Katta in Tips & Tweaks
on 2/15/2006 at 10:56 AM with Comments (49) & TrackPings (13)
Search
Share This
Recent Posts
- November 2008 - Wii/PS3/360 Games
- Microsoft Updates The Zune
- Logitech MX 1100 Mouse Reviewed
- Acer Aspire One Review
- Nvision 08: A Future Of 3D Gaming
- Nvision 08: Audi Navigation
- Nvision 2008: Keynote
- Lenovo Taking IdeaPad S10 Pre-Orders
- MS Office Ultimate Deal Is Back!
- The Official PS3 Wireless Keypad
Categories
- News
- Site News
- Software
- Tips & Tweaks
- Deals
- Podcasts
- Hacking
- Reviews
- Business
- Entertainment
- Hardware
- Gaming
- Autos
- Mobile
- Internet
- Accessories
Archives
- November 2008
- September 2008
- August 2008
- July 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- View All +
Subscribe
Write a Comment.
49 comments.
Pages: « 1 2 [3] Show All
ebauche says
Fortunately my website isn’t really large enough for prefetching to cause too much of an issue (I think) - but I know if people rammed my bandwidth, and I went offline, I don’t think I could easily sort out the cash to pay for more. Ideally I would prefer my bandwidth to be used by people downloading my music than prefetching!
The idea that bandwidth could be used by someone who might not visit those prefetched pages seems crazy.
Cheyne says
Very handy, fasterfox is bad news for us web developers.
http://www.thewebdesignblog.com
Jérémie says
At last, it reads robots.txt. Well, that was much needed.
Ajay D'Souza says
Now that is really good news.
Have added it to my robots.txt Now wait and monitor :)
Any clue on how to check how much bandwidth was it actually eating up?
J says
My websites are update, but most won’t.
Knife In Your Eye says
This is another invention to save our beloved cyberspace… much like segregating nodes via ip address (for the elite), p2p “traffic shaping” (censorship) and use of DRM to (jacking the consumers some more)… Why don’t you twats write a pluggin that will make http requests more effiecent rather then making some crap that limits what people can do… You make the users to be the enemies when in fact the bandwidth providers are the bastards…
Robin says
Thats great that it is finally possible to block prefetching. Before that people could download gigabytes of data that would never be read. As for Daniel and Matt, I ban that type of person from ever entering my websites
Tim says
Prefetch makes web browsing 10x faster easily! I use 1.0.3 & respect those sites that do not want prefetch, but I have never noticed any problems with my own site. (Than again, i’m on University webspace)
robert says
Unfortunately, in some of my tests the robots.txt entry doesn’t stop it…
Daniel says
Good, at least they can bork the robots.txt thing :D
Martin says
Thanks for the info. I was just looking up fasterfox to see if it was worth using. But after reading this i think i will wait the extra half a second to download pages rather than cause problems for some people who spend a lot of their own personal time for the benifit of others. I only checked this out because firefox put a block on the download, maybe they are be responsible.
Mark says
Do you know how to go about discovering installed instances of this plug-in? As in, what if I have some users who have installed this and I don’t want them using it… I didn’t see any additional processes or executables caused by its installation.
Jason Short says
Well, if people are going to ignore the request anyway then webmasters will have to take more drastic measures. We implemented a system where if you grab more than x pages in a 5 second period you are banned. Stopped all that fasterfox traffic. Shouldn’t have to do that. Fasterfox is broken. We had some Firefox users downloading the same pages HUNDREDS of times. We had some users machines that would download 1500 pages in a 5 minute period because of fasterfox… That is just dumb. There is NO WAY a human could view all of them.
David says
The old fasterfox xpi (1.0.0) does work with Firefox 2.0.0.4 if you alter the archive with WinRAR to change the install.rdf version checking.
SiteSucker is still around if you really want everything all at once and don’t care how much damage you do. It’s all about personal responsibility.
Turbo Dude says
I just bumped Fasterfox 1.0.0 to work with Firefox 2.
Now ignoring robot.txt. BWAHAHHAHA!
Anonymous says
The user can easily switch off prefetching. Go to Add-ons in the Tools menu, click Options in the Fasterfox entry in Extentions, switch to Courteous and then back to Custom in the Presets menu, and that is the right place to do this: In the Pipelining menu, Enable Pipelining and Enable Proxy Pipelining, and set Max Pipelining Requests to 10. In Rendering, set Pages In Memory to 10. This procedure should get rid of any prefetching problems and still allow great speed.
Tooez Tooworkaround says
It is too bad that the XPI’s code has released to the general population, now it is a simple matter for anyone with a smidgen of programming knowledge to remove the polite-man check and one rouge group called something like Frozey Crew 200* has released a version that is simply nastier than anything that has come before it that utilizes the entire available bandwidth and when it runs out it seeks other sites that have been somehow selected at ramdom to continue the data pull. Just friggen nasty…Why people have to do crud like that is beyond me, there out to be international laws against those kind of nasty-hacker groups.
Dwayne Charrington says
I just implemented this onto my website. That’ll teach those bandwidth thieves a lesson or two. Also might lower the “Digg Effect” considering that a lot of people who visit from Digg are probably using this extension. When your server is taking a beating from 100,000 people and 20,000 of them are using Fasterfox without this your server is already gone.
- Dwayne Charrington.
http://www.dwaynecharrington.com
jenny says
what a hell are u talking about ???
I didn’t figure it out!!!!!
is it dangeros ??
Jacky says
Since there is a FasterFox fork which ignores the robot.txt I will implement a blocking mechanisim. Care needs to be taken not to block search engines but I think this can be done easily.
- Tell all search engines not to spider a specific directory via robots.txt
- Setup a hidden link on every page, let’s call it VandalTrap in this example
- Run a server side script such as PHP on the index page of the VandalTrap directory. If the page is loaded, the .htaccess file is modified to block the user
This could also be extended to display a informing message but if they choose to install software which circumvents the rules then blocking the IP for 20 minutes should be the best choice.
Pages: « 1 2 [3] Show All
13 trackback / pingbacks:
dotCOM addendums » How To Block Fasterfox Requests (February 17, 2006)techactually » Blog Archive » Blocking Fasterfox requests (February 17, 2006)
WTFDYWMTDAI (February 17, 2006)
The.RSS.Reporter (February 18, 2006)
MacTheWeb » Blog Archive » Fighting Fasterfox (February 18, 2006)
raptxt.it » Come bloccare il pre-fetching di Fasterfox (February 18, 2006)
Creation Robot (February 18, 2006)
The Daily Thing Blog - Blocking Fasterfox To Save Your Sanity and Your Wallet (February 18, 2006)
padawan (February 19, 2006)
Snakey’s WeBlog » Blog Archive » How to block FasterFox (February 22, 2006)
19-03-06 - BRIGADOON………2006 (March 19, 2006)
Programming » How To Block Fasterfox To Save Bandwidth (August 23, 2006)
7 Green-themed/Earth-friendly Sites #2 » BoltPress (June 20, 2008)
Leave a Comment