Smashing the Stack for Fun and Credit Solution



  • Overview

In this assignment you will learn about security vulnerabilities in software. In particular, you will find and exploit memory corruption vulnerabilities in order to better understand how simple programming errors can lead to whole system compromise. You will also examine some common mitigation techniques and analyze exploit code (i.e., shellcode). This assignment is due by no later than 10:30 AM on Thursday, September 20, 2018. See Section 7 for submission details.

  • Getting started

The exercises in this lab will be conducted inside a virtual machine environment to promote a consis-tent experience for everyone in the class and facilitate the grading process. To get started, first down-load and install a copy of Oracle VirtualBox for Windows, GNU/Linux, or Mac OS X.2 Next, download the GNU/Linux virtual machine image that we will use for this project and the Project 1 materials3 from Project 1 materials are also available via Blackboard.

  • Virtual Machine

We will be using a special virtual machine image to carry out this assignment. This will be available both on Blackboard and on the course website projects/

The login information for the two available users is:

Username: user

Password: user

Username: root

Password: root

  • Vulnerability identification and exploitation

Your first objective is to analyze the source code of four programs written in C. Each program has at least one exploitable vulnerability. It will be your task to identify the exploitable vulnerability in each program and construct an attack against each compiled program that will hijack program execution in a meaningful way (as instructed). We encourage you to compile the vulnerable programs using the makefile included with

  • Thanks to Sam Small for creating course materials for this class, including this assignment

2The software and license information is available via the Oracle VirtualBox website.

3I recommend using wget or scp to transfer the Project 1 materials to your virtual machine. To use wget for example, use the

command wget

tgz from the shell in your virtual machine.

1 of 5

CS 526 Information Security

Project 1. We will use this same makefile when testing your attacks. For more information about the project makefile, see Section 6.1.

4.1 Exploitation objectives

Once you have identified an exploitable vulnerability in each program, your goal is to construct an attack that provides you with a root shell.4 For programs generated by vulnerable1.c, vulnerable2.c, and vulnerable3.c, you will accomplish this by injecting shellcode into the vulnerable process. You may construct your attack using C, python, or perl. We have provided Aleph One’s shellcode5 for you to use. It is included in the file shellcode.h and launches a command shell when executed. Using the makefile we have provided, you may notice that vulnerable4.c is compiled with a stack canary and a non-executable stack. Since the stack is non-executable, you will be unable to execute shellcode from the stack. However, you must still find a way to launch a command shell using techniques we have discussed in class.

For each program that we have provided, we ask that you explicitly:

  1. Briefly describe the behavior of the program.

  1. Identify and describe the vulnerability as well as its implications.

  1. Discuss how your program or script exploits the vulnerability and describe the structure of your attack.

  1. Provide your attack as a self-contained program written in C, perl, or python.

  1. Suggest a fix for the vulnerability. How might you systematically eliminate vulnerabilities of this type?

For vulnerable4.c, provide answers to the following questions in addition to those listed above.

  1. What is the value of the stack canary? How did you determine this value?

  1. Does the value change between executions? Does the value change after rebooting your virtual ma-chine?

  1. How does the stack canary contribute to the security of vulnerable4?

  • Shellcode analysis

The file mystery shellcode.h defines a character buffer that contains shellcode. You must analyze this shellcode and comment on its design, what it does, and how it works. To do this, we suggest that you disassemble the shellcode and annotate its disassembly with your own comments. Include the code and your comments in answers.pdf. In completing this task, you may find that http://syscalls. is a helpful resource.6

  • You will run each program’s setuid executable as the non-privileged user named user.

  • See Aleph One’s seminal article Smashing the Stack For Fun and Profit, which is posted on the course webpage and all over the Internet.

6Your class notes will be helpful too.

2 of 5

CS 526 Information Security

Figure 1: An example that demonstrates the use of the project makefile and the successful exploitation of vulnerable1.c.

  • Helpful information

6.1 Using the makefile

The makefile provided with the Project 1 materials is called Makefile. It has 3 targets of interest: all (the default target), setuid, and clean. The default target will compile the source code we have provided. The setuid target must be executed as superuser or root. It copies the compiled programs into /tmp/ and marks the executables as setuid root. When a user launches a program marked as setuid root, it runs with the same privileges as the root user. The clean target removes any existing Project 1 executables from your working directory (but not from /tmp/).7 For an example demonstrating the use of the project makefile, see Figure 1.

6.2 Using ssh

To successfully exploit, use an ssh client such as Putty to connect to the virtual environment and execute your code that way. Please also don’t change/customize the user environment (.bash profile), or else your exploit program may have problems when grading.

6.3 Enabling core dump files

Core dump files can be useful for analyzing the cause of an execution crash. Once enabled, the operating system will create a file named core in your current working directory when a process crashes (e.g., due to a segmentation fault). Used in conjunction with gdb, the core file can give you a complete view of the process state when it crashed. To analyze a crashed process, simply launch gdb with the names of the executable file and its core file as arguments.

  • Warning: Files left in /tmp/ may not survive a reboot of your virtual machine.

Figure 2: If you use ssh to connect to your virtual machine and want to enable core dump fi-les, you must execute ulimit -c unlimited and for setuid executables execute echo 1 > /proc/sys/fs/suid dumpable as root.

You can enable core dump files for processes that crash by using the command ulimit -c unlimited from the command prompt. However, to enable core dump for setuid executables, you should use an additi-onal command (echo 1 > /proc/sys/fs/suid dumpable). If you need assistance enabling core files, see Figure 2 or ask for help via the course discussion board.

6.4 Miscellaneous

You may find it helpful to write an additional program or script that will assist in figuring out the correct input to exploit any of the vulnerabilities. Note that this can be separate from the script/program you will write to exploit the vulnerabilities. If you’re unsure about what is acceptable for this assignment, please don’t hesitate to email us. Be sure to turn in the code for any additional program or script that you write.

  • Deliverables, deadline, and other information

This assignment is due by no later than 10:30 AM on Thursday, September 20, 2018. Please submit your project online via Blackboard.8 You can submit your project as many times as you like (n.b., subsequent submissions overwrite previous ones). The deadline is firm so please do not wait until the deadline to upload your assignment.

For this project you will submit the following files:

The file answers.pdf should contain your responses for each project question or prompt . You may use any word processor or typesetting environment of your choosing in creating this file, but you must submit a valid PDF file. Note that you are not allowed to collaborate with anyone on this project. Also note: we will deduct points from your project for excessive violation of reasonable organization, good grammar, or proper spelling.

CS 526 Information Security




A PDF file containing your responses to project tasks and questions


Your exploit targeting vulnerable1


Your exploit targeting vulnerable2


Your exploit targeting vulnerable3



The input file you used in targeting vulnerable4



A text file containing the user commands used for targeting vulnerable4

other files

Any additional files or programs created to support your efforts (for example a Makefile)

Table 1: Your submission tarball should include the following files.

If there is anything non-standard involved in compiling or running your attack files (or anything that you think it would make the TA’s life easier to know when grading your assignment), please include a README as well with any additional information.

You must upload your assignment as a compressed tar archive.9 First, make a directory with a name that is formatted in the style <username>.proj1 using your Purdue Career Username in place of <username>. For example, if H. G. Wells were in our class, he might make a directory named hgwells1.proj1. Copy all of the Project 1 materials we will need for grading your assignment to this directory and then create a compressed tar archive of the directory formatted as <username>.proj1.tgz, e.g., hgwells1.proj1.tgz. For those of you unfamiliar with this process, we list the commands Mr. Wells would have used in Table 2.




Change current directory to your home directory

mkdir hgwells1.proj1

Make directory hgwells1.projt1

cd hgwells1.proj1

Change directory to hgwells1.proj1


/project1/answers.pdf .

Copy the listed file to the current directory


e /project1/attack1.c/py/pl .


e /project1/attack2.c/py/pl .


e /project1/attack3.c/py/pl .


e /project1/attack4 input .


e /project1/attack4 commands .



Change current directory to your home directory

tar zcvf hgwells1.proj1.tgz hgwells1.proj1

Create a compressed tarball from your directory

Table 2: The commands H. G. Wells used to create his Project 1 tarball.

Once you have created your tar archive, you will likely need to copy it from your virtual machine to your host (or somewhere else) so that you can upload it to Blackboard. If you don’t know how to do this or have problems creating your tar file, please post a message to the discussion board so that someone can assist you. Before submitting your project, we strongly advise that you copy your tar file to a temporary directory and unarchive it10 to personally verify that you’ve created a complete and valid tar file. We will deduct points if your tar file does not follow the conventions we have described.