Use PHP and cURL to retrieve URL contents (with a few options)
Posted by Andy Langton on Monday, 27 July 2015
url_get_contents - a function to retrieve the contents of a remote URL with PHP and the cURL library.[break]Because I've used this function for a few different projects, it's grown to be a bit of a monster, however you can still use it in a pretty basic way, i.e. url_get_contents("http://www.example.com");
I'm not a professional programmer by any means, and I suffer from lack of organisation of common code that I use. So, I'm going to post a few of the things that I hunt for here, to save myself a bit of a search. Hopefully they may be useful to someone else too.
Anyway, here's the function, with comments that should hopefully explain the logic.
<?php # url_get_contents function by Andy Langton: https://andylangton.co.uk/ function url_get_contents($url,$useragent='cURL',$headers=false, $follow_redirects=false,$debug=false) { # initialise the CURL library $ch = curl_init(); # specify the URL to be retrieved curl_setopt($ch, CURLOPT_URL,$url); # we want to get the contents of the URL and store it in a variable curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); # specify the useragent: this is a required courtesy to site owners curl_setopt($ch, CURLOPT_USERAGENT, $useragent); # ignore SSL errors curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); # return headers as requested if ($headers==true){ curl_setopt($ch, CURLOPT_HEADER,1); } # only return headers if ($headers=='headers only') { curl_setopt($ch, CURLOPT_NOBODY ,1); } # follow redirects - note this is disabled by default in most PHP installs from 4.4.4 up if ($follow_redirects==true) { curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); } # if debugging, return an array with CURL's debug info and the URL contents if ($debug==true) { $result['contents']=curl_exec($ch); $result['info']=curl_getinfo($ch); } # otherwise just return the contents as a variable else $result=curl_exec($ch); # free resources curl_close($ch); # send back the data return $result; } ?>
Category:
Comments
Duyen Hoang
Thu, 07/31/2014 - 07:05
thanks alot :)
thanks alot :)
Manoj Chakravarthy
Mon, 02/27/2017 - 22:33
What if the url we are trying
What if the url we are trying to access is a user restricted one? Only users with some special permissions can access.
Thanks in advance.
Add new comment