Chiral Software, Inc. 310-356-7869

Installing JBoss Application Server on Ubuntu Linux

October 31st, 2008

Introduction - hosting applications with JBoss Application Server

Please install a SVG plugin for your browser, or use Firefox to view this image.

In a previous article we showed how to run non-root processes on port 80. Java applications have inherent security advantages because they are running in a virtual machine. We can do even more by not running the server as root, ever. In contrast, a server like Apache HTTPD typically does run at least partly as root, and indeed, there have been exploits that have targeted Apache HTTPD in the time before it drops root. We can avoid this entirely using the iptables port redirection.

In this article, we'll show how to install JBoss as a daemon which starts on system startup and can be managed like the other daemons. We'll also configure logging properly, and secure the JMX console. These are all the tasks necessary to deploy JBoss Application Server as a normal server process, integrated into the Ubuntu Linux environment.

We're on Ubuntu Server 8.04 LTS.

Update, November 14th 2008: These instructions have been updated for Ubuntu Server 8.10 "Intrepid Ibex". In particular, the JBoss init script that worked with Ubuntu 8.04 does not work with Intrepid, and has been updated.

Preparation

Before you begin, make sure that the sun-java6-jdk package is installed.

Create a user which the JBoss AS will run as. We create a separate user with a home directory, but no ability to log in and no password.

useradd -d /usr/local/jboss -s /bin/sh jboss

-d /usr/local/jboss specifies /usr/local/jboss as the home directory. -s /bin/sh specifies a standard shell, which is necessary for executing commands. Not specifying a password also disables login or password verification. You can see in /etc/shadow that the password value for jboss is a single !, meaning no password is valid.

Download JBoss AS. We're using version 4.2.3 for JDK 6.

Pick a place to install the server. Unfortunately the directory structure of JBoss AS is not a great fit for Linux. A proper Linux server application would have the server software itself in one location, probably under /usr. The server's configuration data would be under /etc, and the server's applications would be under /srv, and logs would be placed under /var/log. We can achieve most of that, except it's not feasible to move the configuration files into /etc.

See: good overview of the standard Linux filesystem hierarchy for a good overview of the standard layout for Linux. I hope that future releases of JBoss AS will have better ways of integrating with the Linux directory structure. Even nicer would be the ability to install it properly with apt-get.

For now, we'll put it under /usr/local, as the jboss_init_redhat.sh script suggests. Unzip the zip file there, set the ownership to jboss / jboss, and also create a symbolic link so it can be accessed as /usr/local/jboss.

After unpacking, make sure user and group ownership are set to jboss.

Installing the init script

Copy the RedHat init file to the init directory:

cp /usr/local/jboss/bin/jboss_init_redhat.sh /etc/init.d/jboss

A few modifications are needed. It expects the JBoss server to be installed in /usr/local/jboss so that does not need to be changed, unless you have installed it somewhere else. It runs the server as user jboss which we have done here. These are the changes:

  1. Edit the JAVAPTH line to use /usr/bin as the path. Again, I hope that someday packages will assume that the Java executable is /usr/bin and won't use path specifications. Until then, we need to tell it that java is in /usr/bin, the same place it has been for years and years.
  2. The JBOSS_HOST line must be changed: JBOSS_BIND_ADDR=${JBOSS_HOST:-"-b 0.0.0.0"}. This binds it on all IP addresses. To bind to a specific address, replace the 0.0.0.0.
  3. This script relies on a program called pidof. On RedHat, this is in /sbin but in Ubuntu, it's in /bin, so fix that. Also, RedHat's pidof is slightly different from Ubuntu 8.04's. Ubuntu's requires an -x argument to detect shell script PIDs. Ubuntu's also does not want the same string format. So the line should be for procid in `pidof -x $JBOSS_HOME/bin/run.sh`; do. With that line, it is able to stop the JBoss process correctly.

You can download a copy of the JBoss init script for Ubuntu with all these changes already made.

Add JBoss to the init system correctly

# update-rc.d jboss defaults
 Adding system startup for /etc/init.d/jboss ...
   /etc/rc0.d/K20jboss -> ../init.d/jboss
   /etc/rc1.d/K20jboss -> ../init.d/jboss
   /etc/rc6.d/K20jboss -> ../init.d/jboss
   /etc/rc2.d/S20jboss -> ../init.d/jboss
   /etc/rc3.d/S20jboss -> ../init.d/jboss
   /etc/rc4.d/S20jboss -> ../init.d/jboss
   /etc/rc5.d/S20jboss -> ../init.d/jboss

Test it out by running it

Run JBoss:

/etc/init.d/jboss start

Now the default JBoss console should be available on the host's external interface, on port 80.

Stop JBoss:

/etc/init.d/jboss stop

Before we continue we need to secure the JMX console and all the other web applications JBoss AS starts with.

Putting the applications in /srv/jboss

Web servers are normally putting their applications within either /srv or perhaps /var/www. It's easy to configure JBoss to look in some other directory for its applications.

Edit server/default/conf/jboss-service.xml. Find the org.jboss.deployment.scanner.URLDeploymentScanner mbean. Find the URLs attribute of this. The value of this attribute is a coma-separated list of directories. We use this value:

<attribute name="URLs">
    deploy/,
    /srv/jboss/
</attribute>

Please install a SVG plugin for your browser, or use Firefox to view this image.
WARNING!

Deploying an EAR in some directory other than the deploy directory is not compatible with Hibernate full-text search. For some reason, the scanner that scans the EAR's lib directory does not detect the lucene-core.jar as it is creating the the application's EntityManager. This means that the EntityManager that is created is not a FullTextEntityManager and so all the full text searches fail with ClassCastExceptions:

Exception during request processing:
Caused by javax.el.ELException with message: 
"java.lang.ClassCastException: org.jboss.seam.persistence.FullTextHibernateSessionProxy 
      cannot be cast to org.hibernate.impl.SessionImpl"

If your app uses the lucene-core.jar and a FullTextEntityManager, deploy it in server/default/deploy, not in some external directory. We are investigating this to see if there is some other solution.

Please install a SVG plugin for your browser, or use Firefox to view this image.
The WARs of JBoss

JBoss AS, in its default configuration, has the following web applications present:

  1. deploy/jmx-console.war:
    JBoss JMX Management Console
    The JMX console, server/default/deploy/jmx-console.war
  2. deploy/management/console-mgr.sar/web-console.war:
    JBoss AS Administration Web Console
    The JBoss Administration web console, server/default/deploy/management/console-mgr.sar/web-console.war
  3. deploy/jboss-web.deployer/ROOT.war:
    JBoss AS ROOT.war
    The JBoss ROOT.war application, server/default/deploy/jboss-web.deployer/ROOT.war
  4. deploy/jms/jbossmq-httpil.sar/jbossmq-httpil.war - serves JBossMQ, with a servlet mapping of /jbossmq-httpil/HTTPServerILServlet/*:
    JBossMQ screenshot
    The JBossMQ Servlet, deploy/jms/jbossmq-httpil.sar/jbossmq-httpil.war
  5. deploy/jbossws.sar/jbossws-context.war - deploys at the jbossws context:
    JBossWS screenshot
    The JBossWS Servlet
  6. deploy/http-invoker.sar/invoker.war - this application has no HTML output servlets; it is used to access JNDI resources.

On a live deployed web site, none of these six web applications should be accessible to users over the Internet. The JMX Console and administration web console present obvious security risks. The others provide access which is not necessary or relevant for users. The rule of security is, if it's not necessary, it's prohibited.

We'll go through each of these WARs and secure them.

Securing the JMX console and the administration web console

Both of these are enabled and not password protected by default. This is ok, because JBoss AS only listens on the loopback interface by default. But when it is used in production, it's vital to restrict both of these consoles. The JMX console can be used to issue various commands, shut down the server, and many other things which should not be open. The server administration console can be used to deploy or undeploy applications or other important actions. Neither should be visible to web visitors.

The consoles are ordinary web applications, so they can be secure in the same way any other Tomcat deployed web application is secured.

Assuming we are running the default profile, we need to secure the web applications in:

If we were using the all configuration, we would also need to secure the instances under the server/all directory.

Edit the relevant WEB-INF/web.xml files, and uncomment the security-constraint. Then edit the WEB-INF/jboss-web.xml files and also uncomment the security-domain sections.

Doing this creates two security domain references:

Set passwords for the JMX console: edit server/default/conf/props/jmx-console-users.properties and add username=password pairs. Then edit jmx-console-roles.properties to assign the necessary roles to those users. In particular, the JBossAdmin role is the role to edit for the JMX console.

Set passwords for the administration web console: edit server/default/deploy/management/console-mgr.sar/web-console.war/WEB-INF/classes/web-console-users.properties and web-console-roles.properties in the same way to secure the web console.

Test these. Verify that passwords are needed to access the JMX console and administration web console.

The ROOT.war application

This application is disabled if some other application is deployed which binds to the root context. Therefore the ROOT.war application can be left as-is. Alternatively, the entire WAR directory can be deleted, or a security constraint could be added.

JBossMQ

Unless you have an application running that requires JBossMQ, this application can be simply deleted. JBossMQ is a legacy service, and will be replaced with JBoss Messaging in the JBoss 5.0 release. As of the writing of this article (early November 2008), the JBoss 5.0 release is imminent.

Delete the entire directory:

rm -r  server/default/deploy/jms

You can also delete the JAR file:

rm server/default/lib/jbossmq.jar

JBossWS

This provides access to JBoss web services. Some applications do provide web services with no security constraints on them, such as a web service that provides public information. If you have such a web service in one of your applications, leave JBossWS as it is, because it must be open to the public. Otherwise, web applications that provide web services can provide their own security contraints. If there are no web services needed in the JBoss installation, the entire SAR can be removed. The directory to remove is deploy/jbossws.sar.

invoker.war

This is the legacy style "detached invoker" for doing RMI. EJB3 does not use this. It should be removed.

If you don't want to remove it, you can restrict it. Unless you explicitly want to provide JNDI access to the public, all the URLs in this application should be restricted, or else the application itself should be removed. If no remove JNDI access is needed, go ahead and delete invoker.war. Otherwise, edit the deploy/http-invoker.sar/invoker.war/WEB-INF/web.xml file and change the URLs which are being restricted from /restricted/* to /*.

But if you don't need it, delete the whole invoker. It's not needed:

rm -r deploy/http-invoker.sar

Testing them all

Here is a table of the WARs and their URLs:

WARURL
jmx-console.war /jmx-console
management/console-mgr.sar/web-console.war /web-console
jboss-web.deployer/ROOT.war root context
jms/jbossmq-httpil.sar/jbossmq-httpil.war /jbossmq-httpil/HTTPServerILServlet/*
jbossws.sar/jbossws-context.war /jbossws
http-invoker.sar/invoker.war invoker - see WEB-INF/web.xml for servlets

After you have set up access restrctions on all the WARs, test them.

Configure logging

This step is optional. If you don't change anything logs will go in server/default/log. This isn't bad but it makes more sense for logs to go under /var/log, the same as the other system logs. Create a directory for the JBoss logs:

mkdir /var/log/jboss

We can direct all of the server's log output into this directory using the jboss.server.log.dir parameter. The best place to put that setting is either within /etc/init.d/jboss, as an argument to run.sh, or to edit bin/run.conf. In both cases, the way to set the parameter is to add the following option to the JVM invokation:

-Djboss.server.log.dir=/var/log/jboss

Edit deploy/jboss-web.deployer/server.xml. This is, of course, an ordinary Tomcat server configuration file. Switch from the outdated common log format to the more-useful combined log format. This is also where JBoss virtual hosts are configured, and each can be given its own log prefix. Note that this file refers to the jboss.server.log.dir parameter, so if that parameter is set, the logs will be deposited there.

Finally, JBoss may be logging too much in its default configuration. Edit server/default/conf/jboss-log4j.xml. We use EJB3 in our applications, and EJB3 uses Hibernate as its provider, and we find our logs full of Hibernate query logging. One parameter to check is the hibernate.show_sql parameter in your applications persistence.xml. Setting the parameter to true is helpful for debugging, but generates too much log traffic in real use. You can also add a configuration in the jboss-log4j.xml file to adjust the logging setting of the org.hibernate category:

<category name="org.hibernate">
      <priority value="TRACE"/>
      <appender-ref ref="HIBERNATE"/>
            </category>

This setting sends Hibernate logs to a separate appender, which could be to a different directory. This also turns on TRACE level logging. For a production system you want less logging than that.

For Hibernate to use log4j, the commons-logging.jar must be in the application's WEB-INF/lib.

You can also turn off console logging if you want to by removing the console appender reference from the root block at the end of the file.

What we haven't covered: SSL certificates

Most real-world applications will also need SSL certificates for at least some parts of the site. We'll cover that later in a separate article, where we talk about SSL certificates and also Seam Security.

Conclusion

We have installed JBoss AS in a way which is reasonably well integrated with Linux, using the /etc/init.d mechanisms to start JBoss on server boot. We've also secured some services which need to be secured. We've configured logging in a reasonable way. And don't forget, configure PermGen space correctly, or the server will have a lot of problems if it hosts more than a couple of applications.

It may seem like a lot of effort to install a simple application server, but it's actually quite fast once you have a check-list of the necessary steps.

I do hope that, in future, a few things will be different about the way JBoss AS is distributed:

All in all, Ubuntu Server is a great environment for running JBoss AS. The two together make it possible to build fantastic applications with the greatest security, stability and flexibility in any software platform we have seen.

Update, November 12th 2008:

Ubuntu Server 8.10 "Intrepid Ibex" ships with OpenJDK, so you get a /usr/bin/java in the default installation which provides Java 6. This is more convenient than installing Sun's JDK, which are somewhat cumbersome packages.

And, as of this month, the Linux Standards Base (LSB), which is the standard which allows software developers to create software packages which will install on any LSB-based distribution, includes support for Java 6. At this point, the support is called "trial use", meaning, it is not part of the LSB conformance tests. However, we can expect the major distros to support it, and we can expect it to become mandatory in future versions of LSB. This is good news for anyone shipping a Java application. You can rely on LSB-compliant distros (including Ubuntu and RedHat) to have a Java 6 binary at /usr/bin/java. Finally. Now I hope that all the Linux startup scripts for JBoss, Tomcat, etc, will start relying on LSB and a predictable Java 6 in a predictable location.

Copyright 2003-2010, Chiral Software, Inc. colophon