Diff of two directory trees to create file/folder level patch (incl. binary files)

There's lots of solutions for creating a file level patch for text files and the like. What I'm looking for is an easy script/shell command that will compare oldversion/ newversion/ and give me a...

Are all files in a collection A, accounted for in a differently structured collection B?

If I have two collections of files: A and B (say, two collections of photos). There is an overlap between the two collections (some to all photos in collection A also exist in collection B -...

find files with same name in different directories and count duplicates

I hope you can help me with the following problem. I have 24 directories each containing many (1000's) of files. I would like to find out which combination of directories contains the most number...

Ruby scripting: the %x( command ) and unexpected ( errors

I found a tool to find duplicate files, and now I'm ready to delete the duplicates. I stared at the format of the output file for a bit, and came up with this script. #!/usr/bin/env ruby...

High-throughput viewing and selecting of photos

From of the thousands and thousands of personal photographs in my collection, I'd like to select some special ones to print and display as a collage. All the photos are on one hard drive but...

What if I need synchronicity for users to have a chance to respond to output before giving input?

Imagine implementing an fdupes sort of scenario in Node.js. It seems impossible. What are people's suggestions for this? Take the 'prompt' module on npm. This is roughly what my code looks...

copy all unique files in a directory based on hashes

file=$3 #Using $3 as I am using 1 & 2 in the rest of the script[that works] file_hash=md5sum "$file" | cut -d ' ' -f l #generates hashes for file for a in /path/to/source/* #loop for all files in...

remove duplicate files using fdupes command

I am using linux. I want to delete duplicate files in a directory recurcively. I have 100 files ".html" . i am using fdupes command. fdupes -r -d dirname [1] 9/7.htm ...

escape whitespaces in linux path and file names

i actually cleaning up my system. And as usual i am trying to do it the python way, so i am cleaning up duplicates in my Music library. And now i am trying to find a pattern for re module to...

Execute bash function from find command

I have defined a function in bash, which checks if two files exists, compare if they are equal and delete one of them. function remodup { F=$1 G=${F/.mod/} if [ -f "$F" ] && [ -f "$G"...

Selecting md5sums from a text file and remove duplicates in Linux

I've used the find command and created a file called Duplicates.txt full of the md5sums of a bunch of images. How do I go about finding the duplicate md5's in the file, and then using those to...

Finding files with same size (potential duplicates) in nested sub-folders in Linux Mint shell?

I have used rdfind, fdupes and fslint and have looked at previous posts such as this one. However the solution in the linked post does't help with files scattered in nested sub-folders. rdfind,...

How to find duplicate files in an AWS S3 bucket?

Is there a way to recursively find duplicate files in an Amazon S3 bucket? In a normal file system, I would simply use: fdupes -r /my/directory

How can I create a hash of a directory in Linux in Shell or Python?

What's the easiest way to get a hash-function of a directory in Linux (preferably using shell scripting or Python)? What I'm trying to do is find duplicate subtrees within a large tree of...

Find duplicates of a specific file on macOS

I have a directory that contains files and other directories. And I have one specific file where I know that there are duplicates of somewhere in the given directory tree. How can I find these...

Sed replace a string in the first line of a paragraph

I am trying to automate the periodic detection and elimination of files, using fdupes. I got this beautiful script: # from here: #...

Bash: Find and delete duplicate files from different folders base on name and size of each file

I have merged several old folders of mp3 together via MusicBrainz Picard https://picard.musicbrainz.org/ It did an amazing job identifying mp3 and properly organise them in a structured folder...