Hi,
I used to know how to do this but forgotten.
How can I check the extension of a url ?
You see, sometimes sitemap xml files do not list site urls (.html) for my crawler to extract all the urls.
Instead, they list further xml files and so my crawler has to go one level deep to extract all the site urls (.html).
And so, I now need to teach the crawler to detect the file type once it extracts urls from a page.
Let's say, my crawler extracted this url/link on a page:
https://www.rocktherankings.com/sitemap_index.xml
Now, how to detect the file type or file extension of that url ?
Which php function to use ? I see parse_url() won't do the job.
I really do not want to be using explode() function here and doing things the long way. I remember php has a specific file type function to do things the shorter way to detect the file type.