Efficiently run a loop for every string of an array?

It's hard for me to tell what you're doing exactly, or where you're getting your input from. If it's thousands of arguments I assume it comes from a file?

If so, you can simply do while read -r line; do ...; done < yourfile.

You might also be interested in find and xargs, both of which have functionality built-in to support passing large numbers of arguments to a command.

For example, you could so something like xargs myscript < myfile, and xargs will automatically divide the lines from myfile into the largest possible chunks that can be passed to myscript on the command line. To handle spaces correctly you'd probably want to use -d '\n' if you're using GNU xargs; on BSD/macOS xargs I think you'd need to convert new lines to \0 and use the -0 option.

Or if you have a list of files on the file system you're working with, find -exec's + terminator will give you xargs like behavior (as opposed to the ; terminator, which makes a new call for each argument): find /my/path -exec myscript '{}' '+'

(FYI, the maximum length of command-line arguments is usually about 2 MiB, though it can vary between systems, especially on Linux, where the limit is actually dynamic.)

/r/bash Thread