Use grep or awk to eliminate blank lines and comments

Contents

A configuration file, specifically, the /etc/samba/smb.conf file contains many lines of comments using both the simi-colon (“;“) and the hash (“#“) as well as many empty lines that are either blank or contain whitespace. Using grep to view the file without these lines is one way to do so. To write the correct command can be a bit more challenging. After some trial-and-error, here are the commands.

Environments

  • CentOS 6
  • CentOS 7

grep

The grep command that can be used to display the contents of a configuration file without comment, whitespace lines, and blank lines is this:

grep -v '^;\|#\|^\s$\|^

Some explanation to this somewhat cryptic command:


    

  • The -v is used to select non-matching lines, or could mean exclude these lines from the final output.
    • 

  • The  ^ in front of each pattern means select from the beginning of the line.
    • 

  • The  \ is an escape needed for the | pipe or the pipe will be read as part of the pattern.
    • 

  • The  \s (??) (Note: Sourced below)
    • 



awk


For many years, I preferred working with grep or egrep, until I found this little gem. It is so much cleaner.


awk 'NF' /var/ossec/etc/ossec.conf
awk NF /var/ossec/etc/ossec.conf


Any awk script follows the syntax condition {statement}. If the statement block is not present, awk will print the whole record (line) in case the condition is not zero.


NF variable in awk holds the number of fields in the line. So when the line is non empty, NF holds a positive value which trigger the default awk action (print the whole line). In case of empty line, NF is zero and the condition is not met, so awk does nothing.


grep + awk


The above examples seems to require a different solution.  awk is easier for one than the other.  Where grep is more suited for the other.  Here is the original example with both awk and grep, as using awk alone, left all the # comments. This is still a cleaner solution, and easier on the eyes.


awk NF /etc/samba/smb.conf | grep -v ^#


Source(s)


    

  • http://ubuntuforums.org/showthread.php?t=1725917
    • 

  • https://stackoverflow.com/questions/3432555/remove-blank-lines-with-grep
    • 

  • https://stackoverflow.com/questions/10347653/awk-remove-blank-lines/38219157
    • 


/etc/samba/smb.conf

Some explanation to this somewhat cryptic command:

  • The -v is used to select non-matching lines, or could mean exclude these lines from the final output.
  • The ^ in front of each pattern means select from the beginning of the line.
  • The \ is an escape needed for the | pipe or the pipe will be read as part of the pattern.
  • The \s (??) (Note: Sourced below)

awk

For many years, I preferred working with grep or egrep, until I found this little gem. It is so much cleaner.



Any awk script follows the syntax condition {statement}. If the statement block is not present, awk will print the whole record (line) in case the condition is not zero.


NF variable in awk holds the number of fields in the line. So when the line is non empty, NF holds a positive value which trigger the default awk action (print the whole line). In case of empty line, NF is zero and the condition is not met, so awk does nothing.


grep + awk


The above examples seems to require a different solution.  awk is easier for one than the other.  Where grep is more suited for the other.  Here is the original example with both awk and grep, as using awk alone, left all the # comments. This is still a cleaner solution, and easier on the eyes.




Source(s)


    

  • http://ubuntuforums.org/showthread.php?t=1725917
    • 

  • https://stackoverflow.com/questions/3432555/remove-blank-lines-with-grep
    • 

  • https://stackoverflow.com/questions/10347653/awk-remove-blank-lines/38219157
    •