Tools for cleaning up a site - orphaned files

[DDixon]

Hey Guys,

I'm helping someone I know with a project that involves multiple sites. Each has a messy catalog of old unused images/pdfs/media files etc. The goal is to tidy them up!

I have access to their FTP, and their sites are all built with a custom CMS.

So my options are to go in the 'image' folder on the FTP and make a list of everything ending in .jpg? and then trying to find each of those on the site... or find a tool to automate this!

I found "Xenu Link Sleuth" as it specifically has an 'orphan file' feature.
It also hasn't been updated in 5 years, and I'm a little nervous about it's closed source nature and potential for dubiousness - so I haven't inputted the FTP credentials.

Can anyone help me with alternative options or ideas on how to deal with this?
Or offer their thoughts on Xenu's reliability / safety.

Thanks!

 

[Stephen]

Well, for crawling sites there are a few others, whether they can identify the orphaned files straight off the bat might not be possible, but you could so some quick Vlookup in Excel I guess. Try these ....

ScreamingFrog
BeamUsUp
Deepcrawl (for larger sites)

 

[emp]

Xenu is still good.

You can use it as a crawler, without FTP access.

If you can script, you could tally all *.jpg or *.pdf or *.whatever in the directories and also in the source code of the crawled copy of the sites.

::emp::

 

[DDixon]

Yeah, I did the Xenu crawl was rad, but I was weary of giving it access to this companies FTP without some knowledge of it's safety.

@Stephen thank you for the tool suggestions!

@emp I really need to learn to script! Beginner suggestions?