Monday, January 21, 2008

Break / Bypass download limit !! Hack the squid proxy :-)

Many http proxies enforce a download limit to users.e.g no downloads over 5MB
It is really horrible... if you can download 10 files of 1MB this is simply $%%%

while trying to figure out a workaround I found this very simple method...
why not fool the proxy saying we are downloading only small files ?

Here's a small script to do it. It works with squid proxies.
Download it from here

I wrote its initial version in C. However, it became quite difficult to install because of externel dependancies. But this will work if you have perl & curl which all systems usually have...

If you like this pls link to this page and make this popular... I'd like to see people removing this stupid download limit thing and make the www free !

enjoy breaking limits !!

pls report dead link to indika.udagedara@gmail.com

49 comments:

  1. Dear Sir/Madam
    I like your idea in this project. but do not know how to configure or run it or how does it works. Even i am not gud in perl but i need it because my user account in my college has been blocked because of downloading more than 5omb. Our college squid proxy download limit is 50mb and there is some setting like how much time spent these are also calculated but i am a regular internet user i always uses 4days in a weak in internet how can I get rid from it and tell me detail how to run or how to configure this. My account has already blocked i need it back can i get it.

    ReplyDelete
  2. if i install perl and curl in my windows machine can i use ur script instead of using cygwin.

    ReplyDelete
  3. I'm not sure whether u can install curl on windows. There is perl however.

    there's a configuration section in the script modify the variable $_proxy_auth as to adjust your user:pass

    ReplyDelete
  4. hi ,im from indonesia
    sorry about my bad english.
    i have my local squid on my pc ,that chace_peer ing to my isp squid where limited my download only 15MB .
    how i must implement your perl script to my local squid so my local squid can tell to isp squid that my squid request is not more then 15MB .
    tanks before

    ReplyDelete
  5. Hi,
    Despite initializing the script with a zip file, it continued to insist that I initiaze it whenever I tried to download a zip file, and would refuse to go any further. So I removed the check, and it worked fine :)
    It's downloading the file right now, so don't know whether it'll be successful... The script seems to download the file in roughly 1 mb chunks..
    Brilliant piece of work!!

    ReplyDelete
  6. btw, i'm not a madam :D ... indika is a male name in SL

    ReplyDelete
  7. yes indeed..eg. create a few threads and ask them to download the parts and later concatenate together..

    I was hoping to do that but unfortunately don't have much time :-)

    I'm definitely sure that you can do this elegantly and easily with python

    ReplyDelete
  8. okkk.. i got it working... had to tweak a few http headers...
    Now i am able to download files greater than 5MB... but theres still 1 prob...
    The files i download thru this perl script in the cygwin env.... if i try opening them thru a windows application, i get an error that the file is corrupt .... any idea whats going wrong??

    ReplyDelete
  9. Hi, I'm getting the same error ady was getting where it says:

    Unknown ext in setup.exe. Rerun with -i fileName at C:\prog\pget.pl

    I'm running this on windows. Any idea what to do? He said he modified http headers and I have no idea what headers to modify. pget.ext is in the right directory (at least to my knowledge)
    Thanks

    ReplyDelete
  10. if squid 2.4 can impose a limit it should work too... i dont think they changed the way how this limit is applied from 2.4 to 2.6

    ReplyDelete
  11. Ok this is what I've done. I changed the variable my $_chunkSize = 750*1024;

    So that it downloads 750kb chunks. The problem is that it has downloaded only 750kb of the file. Is there something else I need to change in pget.pl so that it downloads the remaining chunks as well?

    ReplyDelete
  12. Hey, I'm back :P
    I'm having problems.. Like I mentioned, I was having problems with the script asking for initialization of extensions even though I'd initialized it, so I removed that check. But I still initialize it before I download anything. Now, pget.ext contains exe 77, but when I download an exe, the file downloaded doesn't work. It has a generic windows executable icon, and double-clicking on it doesn't do anything.
    Any idea what's wrong?

    ReplyDelete
  13. Hi,
    Did that, it changed the entry to 77, but I'm still getting the same message when i try to download any exe..

    ReplyDelete
  14. I uploaded the script i'm currently using. It works for me in the same scenario as you (perl cygwin).

    If this too fails pls send me a mail with details of your setup/script with ur config/ext file etc.

    ReplyDelete
  15. my campus has capped the download limit ( i can download at a maximum speed of 5 kbps). is there anyway i can bypass this?

    ReplyDelete
  16. Hi Indika,

    Thanks for ur excellent piece of work. It's amazing how powerful these unix utils are. I successfully downloaded a b2z file and I try to download iso image too. But ext for iso is getting the value 0. Where is these values comes from? Is there any kind of list of file types that we can put for ext file rather than feeding it an actual file? what is this process..? I'm sorry I don't have that much of perl knowledge, but I start learning it :D

    Thanks alot again..

    Kal

    ReplyDelete
  17. I'm not sure which proxy. May I know how to check for what type of proxy my company are using?

    And it happened to all file above 10mb. Below that, it works fine

    Thanks

    ReplyDelete
  18. as u said i entered the command "chmod +x pget.pl" in command prompt but it didnt worked. Also i am unable to install curl on my windows. What is the role of cygwin in running the pget.pl??
    I got installed perl on my windows.
    Sry for my very little knowledege about programming.

    ReplyDelete
  19. Ela kiri indika ayye...



    Sujith

    ReplyDelete
  20. Hey, great script, I've made a modification since I noticed some servers consider 0 as the first byte.
    I'll send to your email.

    ReplyDelete
  21. hi abish:
    did u remove the proxy_auth values?
    even if you dont require proxy authentication leave them as they are.

    ReplyDelete
  22. hii myself virinder
    i'm in coll they r using squid proxy with uthentication giving us net only few hours and download limit 20 mb plz help me to solve my prob

    ReplyDelete
  23. is there anyway to accelerate the download speed.. i mean more number of parallel connections?

    ReplyDelete
  24. hi ur script is great..
    i have it all configured but when i download a 200 Mb file with 2Mb chunks after some downloading i get curl: (18) transfer closed with some outstanding read data..
    Plz tell how to fix this

    ReplyDelete
  25. link's not working
    pls help
    thanks

    ReplyDelete
  26. sir
    i followed the instructions but i cant download more then 1 kb even when im downloading file of greater size.
    i have left proxy_auth as it is because we dont reqiure it here.

    PS: cant we download rar file through ur scipt
    thanks in advance

    ReplyDelete
  27. I have done everything as said. But am getting this error:

    ./pget.pl -i trial.zip
    bash: ./pget.pl: /usr/bin/perl^M: bad interpreter: Permission denied

    ReplyDelete
  28. thanks dude.. your script works smart on me.. :)

    ReplyDelete
  29. i tried changing the user:pass to the one used in my download manager.
    anonymous:IEUser@

    I got a request is too large error
    =(

    ReplyDelete
  30. Haha! i managed to download the driver after all!
    just change the ftp:// to http://
    Probably doesnt work with all ftps.
    The download was unstable with lots of pauses. But it came out uncorrupted!!
    Nice job on the script

    ReplyDelete
  31. can u explain how this work?

    ReplyDelete
  32. hi indika,
    your script is great, i have gone through the whole process but there still some problems,
    1.my univercity is using (squid/2.7.STABLE7),
    2. i get a value in exe 73, how can i change it to 77
    3. my domnloaded file is only 4kb,

    is there any solution plz??
    thanks

    ReplyDelete
  33. @sami,
    1. squid version wouldn't matter (i hope)
    2. edit the pget.ext file by hand (not recommended) or use a different exe file and rerun initialization
    3. send me the downloaded file

    ReplyDelete
  34. my univercity squid proxy do not permit ping ,telnet etc services, moreover i can't ping from one department to another department..any bypass solution plz??

    ReplyDelete
  35. i got new solution.
    internet download manager is compatible with cygwin.
    i dont need to make the extension file.
    idm simply grab the file.
    the main point of cygwin is to fool the squid saying its 1 mb right.

    ReplyDelete
  36. i got ERROR: curl is not at usr/bin/curl. Please install or get correct path

    how do i fix this?

    ReplyDelete
  37. Hi indika,, what is the right index (i mean the one in the pget.ext file) for mp4 file,,,

    coz when i run it with -i options, it produced 0 for mp4 file and i can't play my file,,,

    thanks

    ReplyDelete
  38. @rucciva: sorry I don't have a mp4 with me to check. you better try -i with a different file

    ReplyDelete
  39. Hi,
    Someone said...its possible to use Cygwin with IDM Grabber....does anyone know how to do this?
    ...man..i have to compromise big time on download speed with the script...
    but its nice.. xD

    ReplyDelete
  40. Hey indika, thanks before for your script.
    Now I can bypass the download size limit. But, in this comment someone (anonymous) said that:

    "i got new solution.
    internet download manager is compatible with cygwin.
    i dont need to make the extension file.
    idm simply grab the file.
    the main point of cygwin is to fool the squid saying its 1 mb right."

    Is that true? can you tell me how to do that?

    ReplyDelete
  41. i really do not know how that's done :)

    ReplyDelete
  42. hey man...i'm in college and our college transmit net through a proxy server. they setup a restriction that a person can download a file of <20mb at a time, not more than that. if a download large file,,it shows error..request or reply is too large...soo pls help me how to get away from this error and how to check what limitations have been imposed by our college servers on the net....
    any help is highle appreciated..plzzz help!!

    ReplyDelete
  43. Thanks a lot Indika. This works. btw, Do you know of a way to use openssh with cygwin to bypass the squid proxy altogether? I would really appreciate it if you know how since it will allow me to download torrents. :)

    ReplyDelete
  44. Hi Indika,
    I am a n00b at this..n all that u have said sounds Greek to me..I tried googling keywords to understand ur hack but couldnt understand it completely..so can u please comment here, a step-by-step guide mentioning all the software needed on windows 7 to hack squid 2.7/stable 3's download limit.the limit is 95mb right now.i'm using firefox and chrome..in case that's needed..PLEASE.. :( :( thank you in advance....please reply as soon as possible..please :S :(

    ReplyDelete
  45. i installed everything correctly but after that when coming to download or checking the file extension for download it is telling BAD INTERPRETER : PERMISSION DENIED.how can i solve this problem

    ReplyDelete
  46. Hi Indika, I want to know how to change the chunk size in your code. My college has download limit of 50MB. So in each steps I can download up to 45MB. So please say me how to edit the chunk size to 40MB. Thanks!!!

    And do you know any script for downloading torrents. Because my college blocks torrent downloads too...by the your code works like a charm!!!

    ReplyDelete