HW3: More Apache log processing

Answer the following in a Google or Word document (be sure to put your name and date at the top).

1. Make a table with three columns. In the left hand column, make one row for each of these commands:

• cd
• ls
• pwd
• mv
• cp
• grep
• sed
• awk
• cut
• paste
• sort
• uniq
• cat
• less
• more
• du
• head
• tail
• wc
• gzip / gunzip
• bzip2 / bunzip2
• zcat / gzcat / bzcat
• zless / bzless
• tar

Look up each command on the web; you may find some websites do a better job of explaining a command than others, so it's reasonable to look at more than one page if you're not understanding what a command does. In the middle column of each row, write a description, in your own words, of what the command does. In the right most column of each row, include one or more links to the webpages where you found the information.  This will be for your own use, so write it in a way that makes sense to you.

2. Take a look at the following pipeline of commands. What question does it answer? Write your answer in your document. Look up the individual commands and and their flags, then try running it over the Apache log data. You may also find this page helpful on the default Apache access log format Links to an external site. (our logs are the "combined" log format).

NOTE: if you are on macOS, run this command (use gzcat instead of zcat):

gzcat *.gz | \
	cat - *.log *.log.1 | \
	cut -d' ' -f12- | \
	grep -i "bot" | \
	sort | \
	uniq -c | \
	sort -rn | \
	head

If you are on Windows or Linux, run this command (use zcat):

zcat *.gz | \
	cat - *.log *.log.1 | \
	cut -d' ' -f12- | \
	grep -i "bot" | \
	sort | \
	uniq -c | \
	sort -rn | \
	head

When doing these, I'm looking for a good effort, not necessarily the exact right answer. Don't short change yourselves by just trying random things—putting in the time to really give this a good go will pay off in the long run, even if you don't get the exact right answer. You may work on this with others in the class.