This page contains a variety of information for those intending to work on Candlepin and its associated sub-projects.

Eclipse Setup

The Candlepin buildfile has an Eclipse task to generate the .classpath and .project for you.

  • Generate the .classpath and .project files first:

    $ cd candlepin/
    $ buildr eclipse
    
  • Create a Java project as you normally would (File -> New -> Java Project).
  • Choose Create Project From Existing Source.
  • Choose the root of your git checkout.
  • Ensure your M2_REPO classpath variable is pointing to ~/.m2/repository

Now you need to set up Eclipse to conform to our coding conventions. In Eclipse, go to Windows -> Preferences then:

Import Order

  • Go to Java -> Code Style -> Organize Imports
  • Click the Import button and import project_conf/eclipse/candlepin.importorder

Code Formatter

  • Go to Java -> Code Style -> Formatter
  • Click the Import button and import project_conf/eclipse/code_formatter_candlepin.xml

Templates

  • Go to Java -> Code Style -> Code Templates
  • There will be a section for “Comments” and for “Code”
  • For each section, click Import and import project_conf/eclipse/code_templates_comments.xml for comments and project_conf/eclipse/code_templates_code.xml for code.

Checkstyle

  • See instructions here. The .checkstyle file for a project will be generated when you run buildr eclipse.

Code Style

Our canonical reference for code style is the project_conf/checks.xml file which stores all the Checkstyle configuration.

Also see the Java Coding Conventions

For Python, stick to the guidelines in PEP8. Also, run make stylish to run pep8, pyflakes, pyqver, rpmlint, and a few subman specific code checks.

For C, run this on your code before you commit: indent -linux -pcs -psl -ci8 -cs -cli8 -cp0 yourawesomefile.c. Note that you may need to double-check that the “*” is on the right line, and feed in arguments to indent as appropriate.

Committing Code Changes

Commit Messages

General commit messages should follow the following format:

A short one line description of what you did.

Then a newline, and optionally provide any extra information here.

When committing a bug fix from bugzilla (BZ), the following format should be used:

\<Bug Number\>: Short one line description of what was done.

Then optionally a newline and other information.

For example:

712415: Make the names consistent between list --installed and list --consumed

We use these for changelogs when tagging builds. It may seem pedantic but when you need to process a few hundred lines it’s very helpful if they’re typo free, changelog friendly, and have the bz’s automatically detected.

A general git guide can be found here.

Important Notes

  • Please be sure that when committing code, you have your git author info set up correctly, and that you are working as the correct user.
  • Take a few seconds before committing to ensure that your commit messages follow the correct format, and are typos free.

Testing

Testing is extremely important for the team. We have a variety of test suites on the go, all of which should be kept passing before you commit to any given codebase.

  1. Candlepin
    • Java unit tests: Standard JUnit tests which can be run from within Eclipse or from the CLI.

      $ buildr test
      

      A specific suite can be run with:

      $ buildr test:EntitlementCuratorTest
      
    • Functional rspec tests (from /server directory):

      $ buildr rspec
      
    • Functional tests in parallel (from /server directory):

      $ buildr rspec:parallel
      
    • The safest bet, in addition to rspec, is to run check_all (includes all lint tasks, test, validate_translation) before committing:

      $ buildr check_all
      
  2. Subscription Manager
    • Python nosetests: Generally unit tests, which should not require root access or a live Candlepin server to run.

      $ nosetests
      

      Subscription-manager tests need an X server DISPLAY set, since they run gui tests as well. To avoid showing those tests on screen, you can setup an offscreen vncserver and set DISPLAY to point to that server for the tests.

      $ DISPLAY=:13 nosetests
      
  3. python-rhsm
    • Still a very small suite of tests which needs work, and requires a live Candlepin server on localhost. (needs work)

      $ ./setup.py nosetests
      

      The setup.py is required just the first run to compile the C library and copy it to the correct location for the unit tests. After that you can use nosetests normally, although you may need to re-run the above if anything changes in the C code.

  4. Headpin

    $ rake spec
    
  5. Thumbslug: TODO

Mocks

When possible, we try to leverage mocks in unit tests to skip complicated/costly setup of objects we’re not interested in, and instead just focus on testing the component we are interested in. This is a bit of an art form in itself and can be quite tricky to get the hang of, and when it goes wrong you can end up with an un-maintainable mess. Look for good examples, experiment, chat with the team, and in general just try to leverage this when possible. We’re all still learning how this works. :)

In the Java unit tests this is accomplished with Mockito.

In subscription-manager and python-rhsm we use the python-mock module: http://www.voidspace.org.uk/python/mock/

Architecture Gotchas

Candlepin can be a confusing beast. Some pointers that may help to understand how things work and why they are the way they are.

Service Adapters

Central to Candlepin’s design is the use of adapters to abstract services which may or may not be provided by Candlepin components. Objects such as Subscriptions, Products, and Users all may live in external systems depending on the deployment.

Tips:

  • Don’t directly query the curators for these objects, use the service adapters instead.
  • Don’t relate hibernate objects we do store in our database directly to these objects. You’ll have to store the ID instead.

Subscriptions vs Pools

These two objects are almost the exact same thing. They both exist because we may not be the canonical source for Subscription data. As such we use the Subscription service adapter to query subscription data, and use this to create/update/delete our own Pool objects (which are always in our database). The Pool’s are then used to track consumption.

Reading SSL Data for Debugging

See the debugging with wireshark page

Bugs

  • When marking a bugzilla as modified, include a comment with:
    • SHA1 for the commit(s) of the branches your fix went to, be sure to do this after you push, any rebase will change the SHA1.
    • the version of the package the fix will appear in. (look for the most recent tag, and add 1)

Tips

Auto-Generating candlepin.conf

Buildr can auto-generate candlepin.conf for you. This is very useful when you are constantly switching between databases. See the AutoConf page.

Running subscription-manager from a source checkout

To avoid having to “make install” or “make install-files” (install all the code but not the default configs, etc) or installing with tito, you need to make sure bin/subscription-manager can find the code from the source checkout on its PYTHONPATH.

What complicates simpling setting PYTHONPATH is that installed eggs are prepended to the system path before PYTHONPATH. So if a subscription-manager egg is installed, this doesnt work.

One approach to work around this is to use the fact that the local directory is first in the path, so we can make relative imports find the local subscription-manager by symlinking bin/subscription_manager to src/subscription_manager.

# project root
cd subscription-manager

# go into bin/ subdir
cd bin

# symlink bin/subscription-manager to ../src/subscription_manager
ln -s ../src/subscription_manager subscription_manager

With that setup, running sudo bin/subscription-manager from the project root will use the code from the local checkout.

Note that this is only for the application code. plugins will not be found this way, since the code specifically looks for them installed on the system.

Backup / Restore A Database

It can be helpful for developers to save a postgresql database for later use particularly when they’re loaded with a complex or large amount of data.

$ pg_dump -U candlepin candlepin > candlepindb.sql

To restore an old database:

$ sudo service tomcat6 stop
$ dropdb -U candlepin candlepin && createdb -U candlepin candlepin
$ psql -U candlepin candlepin < ~/src/candlepin/candlepindb.sql
$ buildconf/scripts/deploy -g -t

Debugging subscription-manager with eclipse/pydev

Normally, eclipse is kind of useless for debugging subscription-manager since it runs as root, and eclipse does not support that.

However, it is possible to remote debug a root owned process with eclipse.

The setup needs:

With the remote debug setup, subscription-manager can connect to the eclipse debugger server, even if the process is root owner (or, indeed, remote).

So next step is to alter the code to connect to the eclipse/pydev debugger. Somewhere early in the startup process (I usually use the ‘subscription-manager’ or ‘subscription-manager-gui’ scripts) add the following lines, adjusting paths for your eclipse/pydev setup

sys.path.append('/path/to/your/eclipse/installation/eclipse/plugins/org.python.pydev_2.7.1.2012100913/pysrc')
import pydevd
pydevd.settrace()

It can be useful to make a copy of the scripts as ‘subscription-manager-debug’ that include the debug setup code.

With this setup, ‘sudo subscription-manager-debug’ will start the subscription-manager process as root, and try to attach to the pydev remote server. If a server is not running, expect an exception telling you that.

Debugging yum, yum plugins, subscription-manager plugins

Adding the pydevd.settrace() line to /usr/bin/yum will work as expected. However, since it will be using the installed subscription-manager code, locally set breakpoints will not be found. The plugins look for subscription-manager code directly in /usr/share/rhsm.

This assumes ‘yum-debug’ is a copy of /usr/bin/yum with the debugger startup code added.

One simple workaround is to point /usr/share/rhsm to /home/you/path/to/checkout/of/subscription-manager/src via a symlink. This isn’t suggested for a ‘production’ setup, but it’s quick and simple for debugging.

# move installed rhsm modules away
sudo mv /usr/share/rhsm /usr/share/rhsm-real

# create a /usr/share/rhsm-debug that points to your checkout
sudo ln -s /home/you/path/to/checkout/of/subscription-manager/src /usr/share/rhsm-debug

# points /usr/share/rhsm to /usr/share/rhsm-debug
sudo ln -s /usr/share/rhsm-debug /usr/share/rhsm

With that setup, ‘sudo yum-debug’ will start yum, connect to the pydev debugger, and use the code from local checkout for the rhsm modules the yum plugins import.

subscription-manager plugins (/usr/share/rhsm-plugins) will all find the correct local copy of code with this setup.

Note similar setups can be done with winpdb. There are other ways to set source paths, but this is pretty quick and easy.

Debugging with Tomcat

To enable remote debugging in Tomcat, you must pass the JVM values telling it to enable JDWP.

  1. Open /etc/tomcat/tomcat.conf
  2. Add the following to the CATALINA_OPTS variable:

    -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
    

    Now you will be able to connect a debugger to port 8000.

    The -Xdebugger -Xrunjdwp version of enabling the debugger has been deprecated as of Java 5. Use -agentlib instead.

Building RPMs with Tito

Candlepin uses Tito to build the rpms, see here.

Using LogDriver (logging JDBC driver)

To use the logging JDBC driver with Candlepin see the log driver page

Analyzing PostgreSQL Performance

CREATE FUNCTION pg_temp.sortarray(int2[]) returns int2[] as '
  SELECT ARRAY(
      SELECT $1[i]
        FROM generate_series(array_lower($1, 1), array_upper($1, 1)) i
    ORDER BY 1
  )
' language sql;

  SELECT conrelid::regclass
         ,conname
         ,reltuples::bigint
    FROM pg_constraint
         JOIN pg_class ON (conrelid = pg_class.oid)
   WHERE contype = 'f'
         AND NOT EXISTS (
           SELECT 1
             FROM pg_index
            WHERE indrelid = conrelid
                  AND pg_temp.sortarray(conkey) = pg_temp.sortarray(indkey)
         )
ORDER BY reltuples DESC
;
Last modified on 27 October 2017