10/15/2018 4PM: Changed “open” request handling when the (username, postid) pair does not exist in the database
In Project 2, we will develop an online “markdown” editor using Apache Tomcat. Through this process we will learn:
How to develop a Web application using a “traditional” stack, in particular, MySQL and Apache Tomcat (Java servlet)
How to use CSS to format and layout a Web page
In later projects, we will use more “modern” Web development stack, such as MongoDB, Node.js, and Angular, but we want to make sure that everyone has an experience with developing Web applications on a more “traditional” stack. This will help students understand and appreciate why modern frameworks are structured as they are. All development for Project 2 will be done on the “tomcat” container that you created from “junghoo/cs144-tomcat” image in Project 1. Make sure that container still starts and works fine by issuing the following command in a terminal window:
$ docker start -i tomcat
Part A: Learn Apache Tomcat
Before starting to code, first learn how we develop a Web application using Apache Tomcat by going over our tutorial:
Developing a Web Application on Tomcat.
The tutorial teaches how you can develop a Web application using Java Servlet and JSP, and how you can package the set of files needed for your app into a single Web Application Archive (WAR) file for easy deployment on Tomcat.
Part B: Learn JDBC
In order to access MySQL data from a Java program, you will need to use JDBC (Java DataBase Connectivity) API. Go over the following tutorial to learn how to use JDBC to access MySQL in a Java program.
A short tutorial on JDBC
Part C: Implement Markdown Editor and Previewer
Now that you have learned how to develop applications on Tomcat and MySQL, it is time to get your real work done. In Part C, you will need to implement an “online markdown editor” that allows users to save and edit blog posts written in markdown.
Markdown is a lightweight and intuitive markup language originally proposed by John Gruber in 2004 to help people “write using an easy-to-read, easy-to-write plain text format, and optionally convert it to structurally valid XHTML (or HTML)”. Due to its simplicity, markdown has become the de-facto standard for writing readme files on many web sites (e.g., GitHub and BitBucket). In fact, this page was originally written in markdown and has been converted to HTML using Pandoc! Since we will use an “off-the-shelf” markdown parsing library in Java for this project, you don’t have to learn markdown syntax, but in case you are interested, here is a 2-minute introduction to markdown syntax. For your reference, John Gruber’s description of markdown syntax is available here (highly recommended if you want to learn more precise syntax) and the result of recent standardization efforts, referred to as CommonMark, is available here (extremely detailed and precise, yet boring to read).
“Pages” Within Your Application
Your Web site should allows users to create a new post (written in markdown), preview the post (rendered in HTML), and manage existing posts. These tasks are supported through three main pages on your Web site: edit, preview, and list pages
The “edit page” allows editing the title and body of a post.
The page should contain two input boxes
a title <input> box of text type. This text input element must have the ID attribute with value “title”.
a body <textarea>. This textarea element must have the ID attribute with value “body”.
The page should contain four buttons: save, close, preview, and delete. Once pressed,
1. “save” button saves the content of the post to the database and goes to the “list page”.
“close” button goes to the “list page” without saving the current content.
“preview” button goes to the “preview page” (without saving the current content).
“delete” button deletes the post from the database and goes to the “list page”.
The “preview page” shows the HTML rendering of a post written in markdown. The page must have a “close” button. Once pressed, close button goes back to the “edit page” of the post.
The “list page” shows the list of all blog posts saved by the user. The posts in the list should be sorted by their “postid” (a unique integer assigned to a post) in the ascending order. Each item in the list must show:
title, creation, and modification dates of the post, and
two buttons: open and delete. Once pressed,
“open” button goes to the “edit page” for the post. “delete” button deletes the post from the database and comes back to the list page.
The list page should also contain a “new post” button to allow users to create a new post. Once pressed, the button should lead to the “edit page” for a new post.
To help you better understand the function of the three pages on your site and their interaction model, we made an example demo site available at http://oak.cs.ucla.edu/editor/post? action=list&username=junghoo. To encourage you to create your own CSS styles for your site, we removed all CSS styling tags in the demo, so it looks ugly. But you can still get an idea of how the three pages should interact and function. For security concerns, we do not perform any update operation to our database. Even if you “delete” a post, for instance, it won’t disappear from the list.
Note: The three “pages” that we described above may not necessarily be three separate HTML pages. Depending on your implementation, they may be implemented as one or more HTML pages, Java servlets and/or JSP pages. You may consider “three pages” as three classes of page layouts that your application should use.
In developing your application, you should follow the following REST API. First, you should make your application available at the following path:
The path is case sensitive. You must use exactly the same case as above both for the the initial path prefix and the parameters. The parameter “action” specifies one of five “actions” that your site has to take: open, save, delete, preview, and list. The other four parameters, username, postid, title, and body are (optional) parameters that the actions may need. Here is detailed descriptions of the five actions – their functions and required parameters:
required parameters: username and postid
function: return the “edit page” for the post with the given postid by the user
If postid <= 0,
if title and body parameters have been passed, use the passed parameter values as the initial title and body values and return with status code 200 (OK)
otherwise, set missing title and/or body to empty string and return with status code
(OK) If postid > 0,
if title and body parameters have been passed, use the passed parameter values as the initial title and body values and return with status code 200 (OK) otherwise
if (username, postid) row exists in the database, retrieve the title and body from the database and return with status code 200 (OK). otherwise, return with status code 404 (Not found).
required parameters: username, postid, title, and body
function: save the post into the database and go to the “list page” for the user
if postid <= 0
assign a new postid, and save the content as a “new post”
if postid > 0
if (username, postid) row exists in the database, update the row with new title, body, and modification date.
if (username, postid) row does not exist, do not make any change to the database
required parameters: username and postid
function: delete the corresponding post and go to the “list page”
required parameters: username, postid, title, and body
function: return the “preview page” with the html rendering of the given title and body
required parameters: username
function: return the “list page” for the user
Among the five actions, you should take requests via both GET and POST methods for open, preview, and list, while you should allow only POST method for save and delete. This is because save and delete leave significant side effects on the server.
When the server cannot handle a request because the request is “invalid” – for example, the request does not include a required parameter – the server must return with a 4XX response status code, such as “400 (Bad request)” or “404 (Not found)”. The server should never return a 5XX response status code because of an invalid request. Look up this page to
see the list of HTTP response status codes and their meaning. Refer to the HttpServletResponse API to learn how to set the HTTP status code in Servlet.
In case of confusion, you may want to reference our demo site, since it follows the above described API.
NOTE: During our grading, we won’t be doing extensive check for security related issues, but there are two things that we may test:
Your implementation should do the basic input validation. For example, we do expect that students make sure that postid is integer.
Your implementation should use preparedStatement in your SQL queries as a basic protection against SQL injection attack. As we mentioned in our JDBC tutorial, we expect students to follow this industry-wise convention, which cannot be overemphasized as a minimal security measure.
Whatever the user input, it will be important that your implementation won’t flake out (like throwing Java exception messages as the output). All requests need to be handled gracefully, however malformed they may be.
Note on MultiThreading:
Tomcat creates one thread per each request and concurrent data manipulation can be taken care of by MySQL. This makes server-side programming easy since the programmer doesn’t have to worry about concurrency. But in order to do things this way, you need to keep two things in mind:
JDBC API is thread-safe only if each thread uses its own connection. If multiple threads share the same connection, the behavior is unpredictable. Create one DB connection in each servlet request handler, such as doGet() and close it before you return from the handler.
Any servlet request handler must not use global states. Any global state that needs to be shared between multiple requests must be stored and managed by MySQL. In particular,
any servlet request handler, such as doGet() must not reference a class instance variable. It must reference only local variables.
Note on Connection Pooling:
You may think that creating one DB connection per request will lead to too much overhead and wonder whether there is any way to “share” the same DB connection among multiple requests. The proper way to do it is to use connection pooling, but this is beyond the scope of this project. Connection pooling is something that must be used in a production environment, but you MUST NOT use connection pooling in this project since it won’t work on the grader’s machine.
All blog posts that are saved by the users must be stored as a row a MySQL table with the following schema:
modified TIMESTAMP DEFAULT ‘2000-01-01 00:00:00’,
created TIMESTAMP DEFAULT ‘2000-01-01 00:00:00’,
PRIMARY KEY(username, postid)
Hopefully, the meaning of each column would be clear from our earlier discussion. MySQL schema definition is case sensitive, so be careful with the case of your schema definition. Create the table within “CS144” table as the user “cs144”. The first blog post of every user must start with postid=1 and their subsequent posts must be assigned to a linearly increasing postid. Therefore, blog posts by different users may share the same postid, but postid is unique within a single user. In implementing your app, you may need to create tables other than the “Posts” table to store other information needed for your app persistently.
To help you get started, we provide skeleton sample code for this project in project2.zip, which has the following set of files:
+- edit.jsp +- WEB-INF
create.sql: This file has has a sequence of SQL commands that create “Posts” table in the CS144 database. If you need to create any additional tables for your app, you will need to add the SQL
commands to create them in this file.
src/main/java/Editor.java: This is a skeleton Java code for servlet implementation. The provided code simply dispatches the request to edit.jsp page, so you will need to implement
your application logic as part of the Editor class, such as connecting to, retrieving from, and updating MySQL database server, and dispatching the user’s request to an appropriate JSP page.
src/main/webapp/edit.jsp: This is a simple JSP page that contains HTML elements for implementing the “edit page”. You will need to update this page according to our earlier specification.
src/main/webapp/WEBINF/web.xml: This is the Tomcat deployment descriptor file prepared for this project. Currently, it sends any request to /editor to the Java class Editor.
build.gradle and deploy.sh: These are gradle “build script” and the app deployment script that you can use to build your WAR file and deploy it to Tomcat. We will explain these scripts in more detail in the next section.
Gradle build script
As you learned from our tutorial on Tomcat, a Web application needs many files that should be carefully prepared and packaged. To help you focus on coding, not packaging, we included a “gradle build script” that takes care of compilation and packaging. To see how it works, unzip project2.zip into a folder, cd to the folder and run the following command:
$ gradle assemble
This will compile all your Java source files in the src/main/java/ directory (currently, just “Editor.java”), package them together with everything under src/main/webapp directory according to the spec, and create a war file at build/libs/editor.war. Once created, you can copy the war file to $CATALINA_BASE/webapps:
$ cp build/libs/editor.war $CATALINA_BASE/webapps
to deploy it to the Tomcat server. Copy the file, wait for a few seconds (so that Tomcat detects the new war file and sets it up), and access http://localhost:8888/editor/post? action=list&username=test from a browser on your host. You will get a (non-functional) version of the “edit page” that has been generated from the edit.jsp page.
As long as you can use our provided gradle build script to compile your code and create the war file, it is okay for you to know nothing about how the gradle build script works. But if you want to learn more, read one of many online tutorials on Gradle.
Your job now is to write Java code in Editor.java and (optionally) add new JSP, HTML, CSS pages to src/main/webapp in order to implement Online Markdown Editor. If you are not sure how the code in Editor.java works or how to write a JSP page, go over our Tomcat Application Development Tutorial. If you are not sure how to access a MySQL database with a Java code, go over our JDBC tutorial.
Note: You may find it helpful that what is written to System.out/err in Tomcat servlet is (1)
printed on console and (2) written to the catalina.out log filelocated in
$CATALINA_BASE/logs/. To access the directory, you will need the root permission. You can start
a bash shell under the root user by the “sudo bash” command, for example.
Commonmark Java Library
To implement the “preview page”, you need to “compile” a markdown-formatted input into an HTML-formatted output. For this, you can use the commonmark Java library. We have already downloaded the library and made it available in our “tomcat” container. Go over the the README.me file on the library page to learn how to use the library. Roughly, the following Java code
Parser parser = Parser.builder().build();
HtmlRenderer renderer = HtmlRenderer.builder().build();
String markdown = “This is *CS144*”;
String html = renderer.render(parser.parse(markdown));
/* html has the string “<p>This is <em>CS144</em></p>\n” */
will “compile” the markdown text This is *CS144* into HTML text <p>This is
Deployment Script deploy.sh
In project2.zip, we also included our “deployment bash script”, deployment.sh, that automates the entire application deployment process. In particular, it (1) creates tables for our application in MySQL (2) build the “editor.war” file and (3) deploy it to the Tomcat server. Once you finish developing your code, make sure that simply running
properly sets up your application. This script will be used to deploy your application during our grading, so it is extremely important to make sure that this script works without any error.
Part D: Style Your Application Using CSS (Optional)
Once you finish developing the functional part of the online markdown editor, the required part of Project 2 is over. But we encourage you to add CSS styling rules to your app, so that the interface is aesthetically more pleasing. This part, however, is completely optional. As long as your code functionally satisfies our requirements, you will get the full score for this project.
W3.CSS library developed by W3schools (which happens to be a favorite of the author because it is lightweight – only 23KB in size –, relatively easy to learn and use, and adequate for designing simple Web sites. You are welcome to use a third-party CSS framework as long as you make sure that your submission runs on the grader’s machine. You can either include the third-party framework in the submission file or link to the public CSS/JS URL(s) in your HTML output.
To recognize the efforts that you may put into this part of the project, we give you as much as 10% extra credit, if your efforts and dedication to this part is clear from your submission.
Your Final Submission
Your project must be submitted electronically before the deadline through our CCLE Course Website. Navigate to Sections on left of the page, and click on the Project 2 submission section. If you submit multiple times, we will grade only the latest submission.
What to Submit
The zip file that you submit must be named project2.zip, created using a zip compression utility (like using “zip -r project2.zip *” command in the container). You should submit this single file project2.zip that has the following packaging structure.
+- README.txt (optional)
+- edit.png, preview.png, list.png
+- Editor.java (and other java files that you wrote)
+- edit.jsp (and other jsp, css, and html files that you added) +- WEB-INF
We have already explained what most of the above files are earlier, but your submission should include the following additional files:
README.txt: This optional file should include any information that you need to communicate to the grader.
TEAM.txt: This file must include the 9-digit university ID (UID) of every team member, one UID per line. No spaces or dashes. Just 9-digit UID per line. If you are working on your own, include just your UID.
edit.png, preview.png, list.png: You need to take screenshots of your edit, preview, and list pages and include them in the zip file.
Please ensure that your submission is packaged correctly with all required files. Make sure that each file is correctly named (including its case) and project2.zip contains all files directly, not within a subdirectory. Please do not include complied Java class files or WAR file.
Perhaps, the most important requirement of your submission is that the grader should be able to deploy a fully functional version of your Web site just by running ./deploy.sh after unzipping your submission. You may get as small as zero points if the grader encounters an error due to incorrect packaging, missing files, and failure to follow our exact spec.
Testing of Your Submission
Grading is a difficult and time-consuming process, and file naming and packaging convention is very important to test your submission without any error. In order to help you ensure the correct packaging of your submission, we have made a “grading script” p2_test available. In essence, the grading script unzips your submission to a temporary directory and deploy your files to Tomcat to test whether they are likely to run OK on the grader’s machine. Download the grading script and execute it inside the tomcat container:
$ ./p2_test project2.zip
(if your project2.zip file is not located in the current directory, you need to add the path to the zip file before project2.zip. You may need to use chmod +x p2_test if there is a permission error.)
You MUST test your submission using the script before your final submission to minimize the chance of an unexpected error during grading. Again, significant points may be deducted if the grader encounters an error during grading. When everything runs properly, you will see an output similar to the following from the grading script:
$ ./p2_test project2.zip
dropping all tables in CS144 database
building and deploying your application…
Total time: 0.86 secs
Total time: 0.894 secs
Finished deploying your app. Sleeping for 30 seconds for Tomcat to pick it
Requesting http://localhost:8080/editor/post? action=open&username=test&postid=0&title=&body=
This is the response from Tomcat. Make sure that it is what you expect