Many http proxies enforce a download limit to users.e.g no downloads over 5MB
It is really horrible... if you can download 10 files of 1MB this is simply $%%%
while trying to figure out a workaround I found this very simple method...
why not fool the proxy saying we are downloading only small files ?
Here's a small script to do it. It works with squid proxies.
Download it from here
I wrote its initial version in C. However, it became quite difficult to install because of externel dependancies. But this will work if you have perl & curl which all systems usually have...
If you like this pls link to this page and make this popular... I'd like to see people removing this stupid download limit thing and make the www free !
enjoy breaking limits !!
pls report dead link to indika.udagedara@gmail.com
Dear Sir/Madam
ReplyDeleteI like your idea in this project. but do not know how to configure or run it or how does it works. Even i am not gud in perl but i need it because my user account in my college has been blocked because of downloading more than 5omb. Our college squid proxy download limit is 50mb and there is some setting like how much time spent these are also calculated but i am a regular internet user i always uses 4days in a weak in internet how can I get rid from it and tell me detail how to run or how to configure this. My account has already blocked i need it back can i get it.
if i install perl and curl in my windows machine can i use ur script instead of using cygwin.
ReplyDeleteI'm not sure whether u can install curl on windows. There is perl however.
ReplyDeletethere's a configuration section in the script modify the variable $_proxy_auth as to adjust your user:pass
hi ,im from indonesia
ReplyDeletesorry about my bad english.
i have my local squid on my pc ,that chace_peer ing to my isp squid where limited my download only 15MB .
how i must implement your perl script to my local squid so my local squid can tell to isp squid that my squid request is not more then 15MB .
tanks before
Hi,
ReplyDeleteDespite initializing the script with a zip file, it continued to insist that I initiaze it whenever I tried to download a zip file, and would refuse to go any further. So I removed the check, and it worked fine :)
It's downloading the file right now, so don't know whether it'll be successful... The script seems to download the file in roughly 1 mb chunks..
Brilliant piece of work!!
btw, i'm not a madam :D ... indika is a male name in SL
ReplyDeleteoh crap, so sorry :P
ReplyDeleteyes indeed..eg. create a few threads and ask them to download the parts and later concatenate together..
ReplyDeleteI was hoping to do that but unfortunately don't have much time :-)
I'm definitely sure that you can do this elegantly and easily with python
okkk.. i got it working... had to tweak a few http headers...
ReplyDeleteNow i am able to download files greater than 5MB... but theres still 1 prob...
The files i download thru this perl script in the cygwin env.... if i try opening them thru a windows application, i get an error that the file is corrupt .... any idea whats going wrong??
Hi, I'm getting the same error ady was getting where it says:
ReplyDeleteUnknown ext in setup.exe. Rerun with -i fileName at C:\prog\pget.pl
I'm running this on windows. Any idea what to do? He said he modified http headers and I have no idea what headers to modify. pget.ext is in the right directory (at least to my knowledge)
Thanks
if squid 2.4 can impose a limit it should work too... i dont think they changed the way how this limit is applied from 2.4 to 2.6
ReplyDeleteOk this is what I've done. I changed the variable my $_chunkSize = 750*1024;
ReplyDeleteSo that it downloads 750kb chunks. The problem is that it has downloaded only 750kb of the file. Is there something else I need to change in pget.pl so that it downloads the remaining chunks as well?
Hey, I'm back :P
ReplyDeleteI'm having problems.. Like I mentioned, I was having problems with the script asking for initialization of extensions even though I'd initialized it, so I removed that check. But I still initialize it before I download anything. Now, pget.ext contains exe 77, but when I download an exe, the file downloaded doesn't work. It has a generic windows executable icon, and double-clicking on it doesn't do anything.
Any idea what's wrong?
Hi,
ReplyDeleteDid that, it changed the entry to 77, but I'm still getting the same message when i try to download any exe..
I uploaded the script i'm currently using. It works for me in the same scenario as you (perl cygwin).
ReplyDeleteIf this too fails pls send me a mail with details of your setup/script with ur config/ext file etc.
my campus has capped the download limit ( i can download at a maximum speed of 5 kbps). is there anyway i can bypass this?
ReplyDeleteHi Indika,
ReplyDeleteThanks for ur excellent piece of work. It's amazing how powerful these unix utils are. I successfully downloaded a b2z file and I try to download iso image too. But ext for iso is getting the value 0. Where is these values comes from? Is there any kind of list of file types that we can put for ext file rather than feeding it an actual file? what is this process..? I'm sorry I don't have that much of perl knowledge, but I start learning it :D
Thanks alot again..
Kal
I'm not sure which proxy. May I know how to check for what type of proxy my company are using?
ReplyDeleteAnd it happened to all file above 10mb. Below that, it works fine
Thanks
as u said i entered the command "chmod +x pget.pl" in command prompt but it didnt worked. Also i am unable to install curl on my windows. What is the role of cygwin in running the pget.pl??
ReplyDeleteI got installed perl on my windows.
Sry for my very little knowledege about programming.
Ela kiri indika ayye...
ReplyDeleteSujith
Hey, great script, I've made a modification since I noticed some servers consider 0 as the first byte.
ReplyDeleteI'll send to your email.
hi abish:
ReplyDeletedid u remove the proxy_auth values?
even if you dont require proxy authentication leave them as they are.
hii myself virinder
ReplyDeletei'm in coll they r using squid proxy with uthentication giving us net only few hours and download limit 20 mb plz help me to solve my prob
is there anyway to accelerate the download speed.. i mean more number of parallel connections?
ReplyDeletehi ur script is great..
ReplyDeletei have it all configured but when i download a 200 Mb file with 2Mb chunks after some downloading i get curl: (18) transfer closed with some outstanding read data..
Plz tell how to fix this
link's not working
ReplyDeletepls help
thanks
sir
ReplyDeletei followed the instructions but i cant download more then 1 kb even when im downloading file of greater size.
i have left proxy_auth as it is because we dont reqiure it here.
PS: cant we download rar file through ur scipt
thanks in advance
I have done everything as said. But am getting this error:
ReplyDelete./pget.pl -i trial.zip
bash: ./pget.pl: /usr/bin/perl^M: bad interpreter: Permission denied
thanks dude.. your script works smart on me.. :)
ReplyDeletei tried changing the user:pass to the one used in my download manager.
ReplyDeleteanonymous:IEUser@
I got a request is too large error
=(
Haha! i managed to download the driver after all!
ReplyDeletejust change the ftp:// to http://
Probably doesnt work with all ftps.
The download was unstable with lots of pauses. But it came out uncorrupted!!
Nice job on the script
can u explain how this work?
ReplyDeletehi indika,
ReplyDeleteyour script is great, i have gone through the whole process but there still some problems,
1.my univercity is using (squid/2.7.STABLE7),
2. i get a value in exe 73, how can i change it to 77
3. my domnloaded file is only 4kb,
is there any solution plz??
thanks
@sami,
ReplyDelete1. squid version wouldn't matter (i hope)
2. edit the pget.ext file by hand (not recommended) or use a different exe file and rerun initialization
3. send me the downloaded file
my univercity squid proxy do not permit ping ,telnet etc services, moreover i can't ping from one department to another department..any bypass solution plz??
ReplyDeleteDont cheat us
ReplyDeletei got new solution.
ReplyDeleteinternet download manager is compatible with cygwin.
i dont need to make the extension file.
idm simply grab the file.
the main point of cygwin is to fool the squid saying its 1 mb right.
i got ERROR: curl is not at usr/bin/curl. Please install or get correct path
ReplyDeletehow do i fix this?
Hi indika,, what is the right index (i mean the one in the pget.ext file) for mp4 file,,,
ReplyDeletecoz when i run it with -i options, it produced 0 for mp4 file and i can't play my file,,,
thanks
@rucciva: sorry I don't have a mp4 with me to check. you better try -i with a different file
ReplyDeleteHi,
ReplyDeleteSomeone said...its possible to use Cygwin with IDM Grabber....does anyone know how to do this?
...man..i have to compromise big time on download speed with the script...
but its nice.. xD
Hey indika, thanks before for your script.
ReplyDeleteNow I can bypass the download size limit. But, in this comment someone (anonymous) said that:
"i got new solution.
internet download manager is compatible with cygwin.
i dont need to make the extension file.
idm simply grab the file.
the main point of cygwin is to fool the squid saying its 1 mb right."
Is that true? can you tell me how to do that?
i really do not know how that's done :)
ReplyDeletehey man...i'm in college and our college transmit net through a proxy server. they setup a restriction that a person can download a file of <20mb at a time, not more than that. if a download large file,,it shows error..request or reply is too large...soo pls help me how to get away from this error and how to check what limitations have been imposed by our college servers on the net....
ReplyDeleteany help is highle appreciated..plzzz help!!
Thanks a lot Indika. This works. btw, Do you know of a way to use openssh with cygwin to bypass the squid proxy altogether? I would really appreciate it if you know how since it will allow me to download torrents. :)
ReplyDeleteSorry, I don't.
DeleteHi Indika,
ReplyDeleteI am a n00b at this..n all that u have said sounds Greek to me..I tried googling keywords to understand ur hack but couldnt understand it completely..so can u please comment here, a step-by-step guide mentioning all the software needed on windows 7 to hack squid 2.7/stable 3's download limit.the limit is 95mb right now.i'm using firefox and chrome..in case that's needed..PLEASE.. :( :( thank you in advance....please reply as soon as possible..please :S :(
i installed everything correctly but after that when coming to download or checking the file extension for download it is telling BAD INTERPRETER : PERMISSION DENIED.how can i solve this problem
ReplyDeleteHi Indika, I want to know how to change the chunk size in your code. My college has download limit of 50MB. So in each steps I can download up to 45MB. So please say me how to edit the chunk size to 40MB. Thanks!!!
ReplyDeleteAnd do you know any script for downloading torrents. Because my college blocks torrent downloads too...by the your code works like a charm!!!