SAGITTARIUS (Nov 22 - Dec 21)
You are optimistic and enthusiastic. You have a reckless tendency to
rely on luck since you lack talent. The majority of Sagittarians are
drunks or dope fiends or both. People laugh at you a great deal.
In this lab you will be performing the following tasks:
This lab is due at 5:05pm, Thursday February 8th, 2007. The grading criteria for this lab are the following. Points for each part are listed in brackets []:
/etc/init.d/etc/security/etc/security/iptables-rules file has your user-defined SSHD chain as well as the rule for the INPUT chain to allow SSHDFilter to properly function/etc/init.d/iptables stop flushes all rules, deletes any extra user-defined chains, and sets default policy on all three default chains to ACCEPT/etc/init.d/iptables start loads all rules from /etc/security/iptables-rules/etc/init.d/iptables restart performs the two above actions 'stop' and 'start', in that order/etc/init.d/iptables status will list the current IPTables ruleset in verbose mode and without DNS lookups. The two options you need to look at are (-n) and (-v). Check the man page.Usage: iptables {start|stop|restart|status}"/etc/security/iptables-rules does not exist/etc/rcX.d directories (as updated by the update-rc.d script)Let's get right to it. I was asked to do the following in a Google interview: "Write a command that will rename all files in a directory that end in .htm to .html." To understand how to attack this problem, let's look at a few features of our Bash shell.
We've actually already seen an example of 'parenthetical' command insertion. I say 'parenthetical' because I can't think of a better way to describe it. I could say 'commands that are run when put in parentheses and have their output inserted back into the other command'. Yeah, that's a bit long an even obtuse in and of itself. We will work from what we did in the last lab.
I have created a nice script for you that automatically generates a directory tree with files with differing extensions for the purposes of this lab. This way you can easily test these commands and recreate the file structure in case you screw up (and we will as we go). You can get this script here, or alternately directly download it to your machine:
chris@coolname:~$ wget http://www.cs.colorado.edu/~schenkc/htmltree.sh
--04:28:07-- http://www.cs.colorado.edu/~schenkc/htmltree.sh
=> `htmltree.sh'
Resolving www.cs.colorado.edu... 128.138.242.233
Connecting to www.cs.colorado.edu|128.138.242.233|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 295 [application/x-sh]
100%[====================================>] 295 --.--K/s
04:28:07 (5.31 MB/s) - `htmltree.sh' saved [295/295]
You now have to make the script executable and then run it:
chris@coolname:~$ chmod +x htmltree.sh chris@coolname:~$ ./htmltree.sh
Alright, now that we have a basic directory structure setup, we can now dig into the pieces we need to successfully rename files. Let's first go back to our previous lab where we saw parenthetical command subtitution with the uname command.
chris@coolname:~$ echo $(uname -r) 2.6.17-10-server
The above command simply echoes the output from the command uname -r. Let's see what happens when we remove the echo:
chris@coolname:~$ $(uname -r) -bash: 2.6.17-10-server: command not found
This time it actually tried to run a command called 2.6.17-10-server but that program doesn't exist in our path, and most likely never will. There's another way we can perform the same thing, using backticks. Backticks can be found under the tilde '~' key, are not to be confused with 'single quote'
I've read in random places that 'parenthetical' command insertion is meant to replace 'backticks' as the preferred means of output insertion in Bash, but I see backticks used more than anything else. Know that both ways exist and perform the same function.
Let's look at another example of command insertion. We know that we can look at a file's permissions and such using ls -l. I've also previously in class used the command which that tells you what actual program will be run (in your path) for a command typed.
chris@coolname:~$ which vim /usr/bin/vim chris@coolname:~$ ls -l /usr/bin/vim lrwxrwxrwx 1 root root 21 2007-01-17 02:31 /usr/bin/vim -> /etc/alternatives/vim chris@coolname:~$ ls -l `which vim` lrwxrwxrwx 1 root root 21 2007-01-17 02:31 /usr/bin/vim -> /etc/alternatives/vim chris@coolname:~$ ls -l $(which vim) lrwxrwxrwx 1 root root 21 2007-01-17 02:31 /usr/bin/vim -> /etc/alternatives/vim
Command insertion is a rediculously powerful tool for executing commands in Bash. We will use this a little later. But first, let's now look at how for loops work in Bash.
For loops (along with many other things in Bash like 'if' statements which we will see later) can be somewhat cryptic at first because of necessary semi-colons (;) and keywords in places. Let's take a look at a very simple for loop. I use a new command here called file, which tells you what type a file is.
chris@coolname:~$ for i in *; do echo $i; done script sshdfilter-1.4.5 sshdfilter-1.4.5.tar.gz
Let's pick apart the basic structure of a Bash for loop:
% for <variable> in <list>; do <command>; done
Here are a few things to note about the for loops:
<variable> does NOT have to be used in the command.<list> does NOT have to be just files, it can be ANYTHING, such as a list of numbers or whatever.<variable> is a regular bash variable, so when used, you must prepend it with the $$i you must use single quotes '$i' around the variable.Let's actually take a look at an example of single-quoting the $i variable:
chris@coolname:~$ for i in *; do echo '$i'; done $i $i $i
Single quoting is pretty useless in this case, but it can be useful in others. Now let's try to do something more interesting. First: change into the directory that the htmltree.sh script created and let's show all files in that directory (including directories):
chris@coolname:~/htmltree$ for i in *; do echo $i; done file1.htm file2.htm file3.txt file4.html README README.htm subdir1 subdir2 weee.html
Booyeah! We have all our files. Now let's try to list ONLY the .htm files:
chris@coolname:~/htmltree$ for i in *.htm; do echo $i; done file1.htm file2.htm README.htm
Double booyeah! Now we are in a good spot to try to rename our files. Let's quickly look at the usage of mv:
% mv [OPTION] <source> <dest>
Ok, now let's try to rename in our loop. If all we have to do is add an 'l' to the end of each file, let's simply write it that way:
ris@coolname:~/htmltree$ for i in *.htm; do mv $i $il; done mv: missing destination file operand after `file1.htm' Try `mv --help' for more information. mv: missing destination file operand after `file2.htm' Try `mv --help' for more information. mv: missing destination file operand after `README.htm' Try `mv --help' for more information.
Oops. We are missing an operand because the $il variable doesn't exist! Let's quote our variable instead:
chris@coolname:~/htmltree$ for i in *.htm; do mv $i "$i"l; done chris@coolname:~/htmltree$ ls -l total 2 -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file1.html -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file2.html -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file3.txt -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file4.html -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 README -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 README.html drwxr-xr-x 2 chris chris 1024 2007-02-01 04:35 subdir1 drwxr-xr-x 2 chris chris 1024 2007-02-01 04:35 subdir2 -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 weee.html
Booyeah! All files that were .htm are now .html! Now lets try to rename them BACK to .htm This, however, is a little more difficult, and requires the use of 'command substitution' that we looked at earlier. Let's look at why this is difficult.
Our for loop gives us the name of each file. This is all we have to perform a rename. What we just did above was easy because we simply had to add an 'l' on the end of each filename. Now we have to REMOVE that 'l' from each filename, which we can't do without the use of another command. basename will give us the ability to strip the 'l' off the end. Let's take a look at how basename works:
% basename <name> [SUFFIX]
Let's try this out on a filename manually to see what's happening:
chris@coolname:~/htmltree$ basename file1.html file1.html chris@coolname:~/htmltree$ basename file1.html .html file1 chris@coolname:~/htmltree$ basename file1.html l file1.htm chris@coolname:~/htmltree$ basename /home/chris/htmltree/file1.html l file1.htm chris@coolname:~/htmltree$ basename /file/that/doesnt/exist.somewhere where exist.some
Notice in the last example that basename is simply using the string on the command line and the file doesn't actually have to exist. We can now use this in our rename back to .htm:
chris@coolname:~/htmltree$ for i in *.html; do mv $i $(basename $i l); done --OR-- chris@coolname:~/htmltree$ for i in *.html; do mv $i `basename $i l`; done -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file1.htm -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file2.htm -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file3.txt -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 file4.htm -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 README -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 README.htm drwxr-xr-x 2 chris chris 1024 2007-02-01 04:35 subdir1 drwxr-xr-x 2 chris chris 1024 2007-02-01 04:35 subdir2 -rw-r--r-- 1 chris chris 0 2007-02-01 04:35 weee.htm
Take a look at that shizzle. Now all five files that did end in .html now end in .htm. Remember I said that the <list> doesn't have to be files. Let's look at another example using the seq command, which can be really useful for moving files that may have indices within the name (file1.html, file2.html, file3.html, etc):
chris@coolname:~/htmltree$ for i in `seq 1 10`; do echo file"$i".html; done file1.html file2.html file3.html file4.html file5.html file6.html file7.html file8.html file9.html file10.html
Now that you have your hands wet in a little Bash scripting, we're going to continue with more to create an init script that will live in the /etc/init.d directory to load your IPTables rules on startup as well as clear them if needed.
Our init script needs to accept one command line option that can be one of the following:
start - Loads our IPTables rules from the file /etc/security/iptables-rulesstop - Flushes all rules, deletes all user-defined chains, and sets the default policy on the three default chains (INPUT, OUTPUT, and FORWARD) to ACCEPTrestart - Performs a stop and start, in that orderstatus - Lists the current IPTables rules without DNS lookups (-n) and verbose (-v)Other requirements for the script are:
/etc/security/iptables-rules file does not exist./lib/lsb/init-functions) available for all init scripts to use. This will log all actions to the file /var/log/boot so we will know whether or not our firewall rules were successfully loaded or not by simply checking the log file after boot.We will be looking at the following scripting constructs in order to create our script:
And finally we will be using another script on the system to install our script to different runlevels as well as creating a simple initial iptables-rules
Create a file (probably in your home directory) to create your script that can be tested before we finally put it in place in /etc/init.d.
Sha-bang goes at the top of your script. What is it exactly? It's the line that tells your shell which interpreter to run for your script. Let's take a look:
#!/bin/bash ...Any interpreter can do in that spot. The shell will read that first line and will execute that interpreter with that script as an argument. Other common intepreters used are for Perl (
/usr/bin/perl), Python (/usr/bin/python), PHP (/usr/bin/php), and others.
Notice the beginning '#'. This is a comment. All comments begin with '#', and do not have to be at the beginning of line. Now that we have our Sha-bang, let's look at Variables.
Variables are simple, but still do have a strict syntax. Strings must be put in quotes, and again the quotes rules are still in effect (single-quotes==no variable expansion, double quotes==variable expansion). If you omit quotes, the script will attempt to execute the stuff after the '=' and send output to the variable. Also, spaces are very important! We do NOT want spaces when setting variables. We can first play with this on the command line. Let's take a look:
chris@coolname:~$ VAR = ls -bash: VAR: command not found chris@coolname:~$ echo $VAR chris@coolname:~$ VAR= ls htmltree htmltree.sh script sshdfilter-1.4.5 sshdfilter-1.4.5.tar.gz chris@coolname:~$ echo $VAR chris@coolname:~$ VAR=ls chris@coolname:~$ echo $VAR ls chris@coolname:~$ $VAR htmltree htmltree.sh script sshdfilter-1.4.5 sshdfilter-1.4.5.tar.gz
Now let's utilize variables in our script:
#our script return value RETVAL=0 #the path to our iptables command IPTABLES="/sbin/iptables" #the path to our iptables-rules file IPTABBLES_RULES="/etc/security/iptables-rules"
The following page has a good lot of information on if statements: http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html
Normally we're used to some simple constructs like if (test) { do stuff; }. In Bash, things aren't quite so easy. Since Bash deals with untyped variables, we have to explicitly use different types of comparitors to do specific tests.
Now, this is all fine and la-dee-dah-dandy, but you can get tripped up 60 different ways from Sunday trying to get these 'if' statements syntactically corrent. Let's look at a simple example trying to tell us whether or not we are root:
chris@coolname:~$ if [$UID != 0] then echo "You are NOT root."; fi -bash: syntax error near unexpected token `fi' chris@coolname:~$ if [$UID != 0 ]; then echo "You are NOT root."; fi -bash: [1000: command not found chris@coolname:~$ if [ $UID != 0]; then echo "You are NOT root."; fi -bash: [: missing `]' chris@coolname:~$ if [ $UID != 0 ]; then echo "You are NOT root."; fi You are NOT root. chris@coolname:~$ if [ $UID == 0 ]; then echo "You are root."; fi chris@coolname:~$ if [ $UID == 0 ]; then echo "You are root."; else echo "You are NOT root."; fi You are NOT root.
Notice how missing semicolons ';', leaving off the ending 'fi', or not getting the right spacing can totally screw you up. This is highly annoying, but good to learn. Most constructs in Bash require an ending identifier which is merely the original identifier reversed (i.e. 'if' with 'fi'). We'll see that later on as well when we look at 'case' statements.
We can actually add more commands in between the 'if' and the 'else'. Check it out:
chris@coolname:~$ if [ $UID == 0 ]; then echo "You are root."; else echo "You are NOT root."; ls; fi You are NOT root. htmltree htmltree.sh script sshdfilter-1.4.5 sshdfilter-1.4.5.tar.gz
If we wanted to make this pretty inside of a script, we would lay this out like follows:
if [ $UID != 0 ]; then echo "Hey hosehead, you aren't root." echo "...just kidding." exit 1 fi
We can actually go futher with 'if' statements. They have a number of ways to test files, in fact, which we could use to test if a file exists. For more information about these file tests, check out table 7.1 on this page.
if [ -f /etc/sshdfilterrc ]; then echo "File exists" else echo "File does NOT exist" fi
Does a directory exist?
if [ -d /home/chris ]; then echo "Directory exists" else echo "Run home to mama" fi
A nice feature of Bash (and all other shells as well) is that you can execute other scripts within your own, but also grab any functions that those scripts may have declared. This way you can write commons functions for things and use them in many scripts by simply 'sourcing' the script. In fact, this is exactly what the Ubuntu crew has done with functions common to all init scripts, and has put them in a file called /lib/lsb/init-functions. Let's try this out in a script:
#!/bin/bash #source our common init-functions script . /lib/lsb/init-functions #use included log functions log_begin_msg "Hello suckas!" log_end_msg 0
It's that easy! Now you have a mechanism to log to the /var/log/boot file whenever this script is run on bootup. It also does some cool stuff like put '[ ok ]' at the end of line. For more examples of hose these init functions are used, you have plenty of other scripts to look at under /etc/init.d
This should be another familiar construct in general programming, but again we have to pay close attention to syntax:
case "$UID" in
0)
echo "You are root!"
;;
*)
echo "You are not root"
;;
esac
Notice how each 'case' is terminated with a double-semi-colon ';;'. The 'default' case is handled as an asterisk '*'. I mentioned in the lab grading requirements that you must print 'usage' of the script if incorrect command line arguments are sent to the script. How can we do that? Turns out command-line arguments are nicely placed into incrementing numeric variables. Let's say I create a script called 'args.sh' that looks like the following:
#!/bin/bash #zeroth argument is the script name echo "script: $0" #first argument is first command line value echo "1st arg: $1" #second argument is second command line value echo "2nd arg: $2"
Let's see what the output looks like when invoked in different ways:
ris@coolname:~$ ./args.sh script: ./args.sh 1st arg: 2nd arg: chris@coolname:~$ ./args.sh 1starg 2ndarg script: ./args.sh 1st arg: 1starg 2nd arg: 2ndarg chris@coolname:~$ ./args.sh 1starg "2ndarg with spaces" script: ./args.sh 1st arg: 1starg 2nd arg: 2ndarg with spaces
Now that we know how to grab our command line arguments, we can properly create a case statement on '$1', the first argument:
#!/bin/bash
case "$1" in
hello)
echo "Hello to you too!"
;;
goodbye)
echo "Sorry to see you go."
;;
*)
echo "I'm deaf and can't hear you."
;;
esac
Let's look at output of this one:
chris@coolname:~$ ./hello.sh hello Hello to you too! chris@coolname:~$ ./hello.sh goodbye Sorry to see you go. chris@coolname:~$ ./hello.sh barf I'm deaf and can't hear you.
A language wouldn't really be much of a language without functions, would it?Let's take a look at how they're created.
function-name () {
echo "Do stuff."
return 0
}
Pretty simple. You can actually pass arguments to the functions, but for our script we won't need to do so. Return values are good to know about, however. You can see the return value of any function or command by looking at the '$?' variable immediately after a command is executed. Let's edit our above 'hello.sh' script and use return values.
#!/bin/bash
RETVAL=0
case "$1" in
hello)
echo "Hello to you too!"
;;
goodbye)
echo "Sorry to see you go."
;;
*)
echo "I'm deaf and can't hear you"
RETVAL=1
;;
esac
exit $RETVAL
And the return value from each invocation:
chris@coolname:~$ ./hello.sh hello Hello to you too! chris@coolname:~$ echo $? 0 chris@coolname:~$ ./hello.sh goodbye Sorry to see you go. chris@coolname:~$ echo $? 0 chris@coolname:~$ ./hello.sh sunovabitchin I'm deaf and can't hear you chris@coolname:~$ echo $? 1
Now, the real question is, how can we use these in functions? Let's try it out:
#!/bin/bash
RETVAL=0
#our new handy dandy function
file_exists () {
if [ -f /home/chris/testfile ]; then
return 0
else
return 1
fi
}
#call the function
file_exists
RETVAL=$?
if [ $RETVAL == 0 ]; then
echo "File exists."
else
echo "We're digging in the wrong place."
fi
exit $RETVAL
Now let's run it:
chris@coolname:~$ ./retval.sh We're digging in the wrong place. chris@coolname:~$ echo $? 1 chris@coolname:~$ touch testfile chris@coolname:~$ ./retval.sh File exists. chris@coolname:~$ echo $? 0
Sweet! Functions, return values, can life get any better. Yes it can, but probably not because you know about functions. Well, maybe a little. Anyway! Moving on.
One other piece of information you need for your lab is to properly list your IPTables rules as they exist at that moment. This is pretty simple to do, let's take a look:
chris@coolname:~$ sudo iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination SSHD tcp -- anywhere anywhere tcp dpt:ssh Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain SSHD (1 references) target prot opt source destination DROP tcp -- custmx.aplogistique.net anywhere tcp dpt:ssh DROP tcp -- server227.qualispace.com anywhere tcp dpt:ssh
Check out the two drop rules at the bottom in my SSHD chain. The DNS names appear instead of IP addresses! This is actually not a good thing, because if your DNS is improperly setup, you won't be able to look up the names, and you will sit there for a long time until DNS times out and then gives you the rules. Instead, we will completely circumvent this with another option (-n):
chris@coolname:~$ sudo iptables -L -n Chain INPUT (policy ACCEPT) target prot opt source destination SSHD tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain SSHD (1 references) target prot opt source destination DROP tcp -- 195.154.137.17 0.0.0.0/0 tcp dpt:22 DROP tcp -- 216.185.43.227 0.0.0.0/0 tcp dpt:22
To be even cooler, we're going to make this output even more verbose because it doesn't contain everything that I like to see (like number of packets that have hit that rule). To do this, we just need to add the verbose option (-v):
chris@coolname:~$ sudo iptables -L -n -v
Chain INPUT (policy ACCEPT 49955 packets, 4590K bytes)
pkts bytes target prot opt in out source destination
19366 1534K SSHD tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 13903 packets, 1980K bytes)
pkts bytes target prot opt in out source destination
Chain SSHD (1 references)
pkts bytes target prot opt in out source destination
3 180 DROP tcp -- * * 195.154.137.17 0.0.0.0/0 tcp dpt:22
3 180 DROP tcp -- * * 216.185.43.227 0.0.0.0/0 tcp dpt:22
For the stop case in your script, you need to know how to do the three following things in IPTables:
To figure out how to do each of these, man iptables and look at the options '-F', '-X' and '-P'.
It's probably a good idea to know how to load rules from a file. The command is quite simple:
chris@coolname:~$ sudo iptables-restore < /etc/init.d/iptables-rules
The command reads rules from STDIN. There are two options for this command which you can see in the man page. These options can be useful depending on what you're doing. For the purposes of our lab, we don't need to worry about them, but it never hurts to know what your commands can do.
Just for consistency, I'll mention a second command which will allow you to save your current IPTables chains and rules to a file. Check it out:
chris@coolname:~$ sudo iptables-save > /tmp/current-iptables.rules
This can be VERY useful if you've created a bunch of rules at runtime manually that you would like to save. In fact, this is the very way that I created the rules file I give you below in the next section by saving the rules after I manually entered them in.
You should now have all you need to write your init script. Be sure to thoroughly test your script with my grading criteria at the top of this lab before you install your script to /etc/init.d. A broken script is of no use to us, so get it right.
There are many examples of init scripts in the init.d directory. The ones I recommend looking at are /etc/init.d/ssh and /etc/init.d/cron.
This part is easy. Once you have your script ready to roll, you can simply copy it into /etc/init.d:
chris@coolname:~$ sudo cp iptables /etc/init.d
Now that your script is in place, we need to enable this script to run at all the runlevels. Luckily for us, there's already an existing script to do this for us:
chris@coolname:~$ sudo update-rc.d iptables defaults Adding system startup for /etc/init.d/iptables ... /etc/rc0.d/K20iptables -> ../init.d/iptables /etc/rc1.d/K20iptables -> ../init.d/iptables /etc/rc6.d/K20iptables -> ../init.d/iptables /etc/rc2.d/S20iptables -> ../init.d/iptables /etc/rc3.d/S20iptables -> ../init.d/iptables /etc/rc4.d/S20iptables -> ../init.d/iptables /etc/rc5.d/S20iptables -> ../init.d/iptables
And that's all it takes! However, we're not quite done. We still need to add in a rules file to /etc/security/iptables-rules. Let's go create that file now and add some rules:
*filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] :SSHD - [0:0] -A INPUT -p tcp -m tcp --dport 22 -j SSHD COMMIT
The first four lines simply set the default policy on the three chains (INPUT, FORWARD, OUTPUT) to ACCEPT, so all packets can flow freely in and out of our system. Notice the two lines above for SSHD. The first one :SSHD - [0:0] adds in a new user-defined SSHD chain. The second line adds a rule for all packets destined for port 22 to be checked in the SSHD chain. We will go over all of these things in another lab, but after this lab, we are ready to start tackling firewall configurations!
You have now completed Lab 02. Good work. Be sure to test your stuff to make sure it works!