TIL in bash scripts IFS=$'\n'

Be extremely careful with this approach! Filenames are sometimes weird and unexplained!

First thing to remember, on many systems, a filename can contain anything except a null character (character 0) or a slash ('/'). This means it can contain spaces (' ') but also tabs ('\t') and even more strangely, a newline character ('\n')

So, you need to think very carefully when you iterate/loop over files.

for file in $(ls)

So, why do I think this this bad.

Let's make some test files:

echo file1 > 'file 1.txt'
echo file2 > 'file[CTRL-V][TAB]2.txt'
echo file3 > 'file

Now lets loop over them:

for file in $(ls) ;do
   echo "File: :$file:"

We see:

File: :file:
File: :2.txt:
File: :file:
File: :3.txt:
File: :file:
File: :1.txt:

Now let's try using the IFS trick: IFS=$(echo -en "\n\b") File: :file 2.txt: File: :file: File: :3.txt: File: :file 1.txt:

This time, it's correctly spotted the tab in file2, but it's split file\n3.txt into two files.

What are some common trick for safely handling files?

Firstly, let the shell enumerate the files for you.

for file in * ; do
  echo "file :$file:"
file :file  2.txt:
file :file
file :file 1.txt:

This does have a risk, however. If no files exist, then the '*' is taken literally:

file: :*:

You can use a bash extension to prevent this happening. It means * (or any glob) converts to no value if it matches no files.

set -o nullglob

Another trick is to use the null character as a separator.

find . -type f -name '*.avi' -print0 | while IFS="" read -d'' -r file ; do

find's -print0 option means that rather than adding a newline character between files it finds, it will instead use a null character. Setting shells IFS to "" and using read's -d '' means it will recognise this null character as the field separator. Now you're free to work with $file safely regardless of any strange characters it might contain.

Last point, quoting variables

$name='file 1.txt'
ls $name
ls "$name"

What's the difference? $name is subject to "expansion" using IFS. If you're changing and saving the IFS value this expansion changes. This could split $name into two or more parameters to a command.

If the variable value is quoted, however, it's ALWAYS treated as one value and doesn't undergo extra expansion like that.

/r/UnixProTips Thread