Use PHP and cURL to retrieve URL contents (with a few options)
url_get_contents - a function to retrieve the contents of a remote URL with PHP and the cURL library.Because I've used this function for a few different projects, it's grown to be a bit of a monster, however you can still use it in a pretty basic way, i.e. url_get_contents("http://www.example.com");
I'm not a professional programmer by any means, and I suffer from lack of organisation of common code that I use. So, I'm going to post a few of the things that I hunt for here, to save myself a bit of a search. Hopefully they may be useful to someone else too.
Anyway, here's the function, with comments that should hopefully explain the logic.
<?php
# url_get_contents function by Andy Langton: http://andylangton.co.uk/
function url_get_contents($url,$useragent='cURL',$headers=false,
$follow_redirects=false,$debug=false) {
# initialise the CURL library
$ch = curl_init();
# specify the URL to be retrieved
curl_setopt($ch, CURLOPT_URL,$url);
# we want to get the contents of the URL and store it in a variable
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
# specify the useragent: this is a required courtesy to site owners
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
# ignore SSL errors
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
# return headers as requested
if ($headers==true){
curl_setopt($ch, CURLOPT_HEADER,1);
}
# only return headers
if ($headers=='headers only') {
curl_setopt($ch, CURLOPT_NOBODY ,1);
}
# follow redirects - note this is disabled by default in most PHP installs from 4.4.4 up
if ($follow_redirects==true) {
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
}
# if debugging, return an array with CURL's debug info and the URL contents
if ($debug==true) {
$result['contents']=curl_exec($ch);
$result['info']=curl_getinfo($ch);
}
# otherwise just return the contents as a variable
else $result=curl_exec($ch);
# free resources
curl_close($ch);
# send back the data
return $result;
}
?>
03.06.2008. 22:17
answerg said on 29.10.2008. 13:49
thanks a lot.
Eligio said on 24.03.2009. 20:52
Is there a way to use this function to duplicate a site then used owned url instead on the original?
Andy said on 24.03.2009. 22:08
You mean like a mirror or web proxy? It's certainly possible to use curl to do that, but might be stretching it a bit.
I don't think it would be too difficult to do (the function above wouldn't necessarily help though).
There are pre-made scripts for that sort of thing anyway. I found the below, for instance, searching Google for [php web proxy]:
http://sourceforge.net/projects/php-proxy/
What eligio wants is to clone/rip off/copy an existing site so he doesn't have to build his own.
Cheap scammer. Internet garbage.
Recent Responses:
Page last (manually) updated: December 23, 2009.
Questions, comments, insults or praise? Have your say: