How to Install Nvidia Digits

Here we discuss how to install Nvidia Digits. This is mostly intended as a documentation of the process I had to go through to install it in my lab environment on a single stand-alone machine housing 3 gpus.

What is Digits?

I encourage you to look at their website and documentation [link1, link2]  : )

Ultimately, it is a nice user interface for training and examining deep learning models. I have only been using it for a few days, but its something that I myself would have developed if I had an infinite amount of time. They seem to be offering all of the functionality that I have in a hodgepodge collection of scripts in a nice stable UI. I can easily say, as well, that its greatly improved my organization being able to review old convergence graphs and models.

 

digits-image1 digits-image2

Components

There are two components to install. Digits is simply a UI that sits on top of Caffe. The version of Caffe that is required isn’t the one commonly available in the BVLC repo, but a Nvidia fork that we’ll call “caffe-nv”. Its dependencies, compilation, and installation are similar to that of Caffe, so we won’t go into full detail on that (though if you need additional assistance, I’ve written a blog post about that here ).

Installing Caffe

At the time of this writing (December, 22nd 2015), the current version of caffe-nv is compiled against a cudnn v4, which has not been released publically as of yet [see https://github.com/NVIDIA/caffe/issues/85]. As a result, 0.14.0-rc.1 will compile, but 0.14.0-rc.1+ won’t.

These are the commands I used to get it installed. If someone has already installed it, you do not need to do this step, and can simply point Digits at the appropriate directory:

  1. git clone https://github.com/NVIDIA/caffe.git caffe-nv
  2. cd caffe-nv
  3. git checkout tags/v0.14.0-rc.1
  4. cp ~/caffe/Makefile.config .
  5. # I took this config from a working version of caffee I have compiled : )
  6. printf "\nLIBRARY_NAME_SUFFIX := -nv\n" >> Makefile.config
  7. make –j20

In the end, we have an installation of caffe-nv in /opt/caffe-nv

Installing Digits

In general, its much easier for each user to have their own Digits installation so that the jobs and models are separated. Since users may not have root permissions to install the necessary python packages, we’ll use a virtual environment so that this tutorial is universal. We need python 2.7, so we’ll specify that on the command line:

  1. virtualenv -p /usr/bin/python2.7 venv2.7
  2. source venv2.7/bin/activate

Now we’re inside of the virtual environment (prefixed by venv2.7):

  1. (venv2.7)tester@ccipd-tesla-001:~$

We clone the digits repo and then go into the digits directory:

  1. git clone https://github.com/NVIDIA/DIGITS.git digits
  2. cd digits

Remember one needs to run the activate command every time the virtual environment is to be used.

Compiling Pip Wheels 

If you have a kind system administrator (of which I am one), you can skip this section and go onto the next one : ) Here we discuss how to setup pip wheels to speed up the installation process. I’ve already pre-compiled and placed the necessary wheels in /opt/wheels using:

  1. pip install --upgrade pip
  2. pip install wheel
  3. pip wheel --find-links=/opt/wheels --wheel-dir=/opt/wheels -r requirements.txt

this actually crashes because scipy needs numpy, which isn’t installed yet, and h5fy needs cython which isn’t installed yet. So to compensate, after the first round of wheels were made, I installed the 2 dependencies and then made the other 2 wheels. Also scikit-image was missing so I’ve added that

  1. pip install --use-wheel --find-links=/opt/wheels --no-index numpy cython
  2. pip wheel --find-links=/opt/wheels --wheel-dir=/opt/wheels -r requirements.txt
  3. pip wheel --find-links=/opt/wheels --wheel-dir=/opt/wheels scikit-image

which resulted in a complete set of wheel in the opt directory.

Installing Wheeled Packages

Regardless if you’ve skipped the previous section or not, you need to install the compiled wheels. It is now possible to quickly install them into the virtual environment by using:

  1. pip install --upgrade pip
  2. pip install wheel
  3. pip install --use-wheel --find-links=/opt/wheels --no-index -r requirements.txt
  4. pip install --use-wheel --find-links=/opt/wheels --no-index scikit-image

Running Digits

We’ll use the production environment version which has better speed and is more stable.

This is straight forward, in the digits directory we simply type:

  1. ./digits-server

It will ask for the location of the caffe-nv which we can provide (/opt/caffe-nv) and it should be off to the races.

If at anytime you want to reconfigure digits, you can use:

  1. python -m digits.config.edit -v

which may be important if you want to point it at a different instance of caffe or to change the dataset/job directories (i.e., you have a large storage volume mounted somewhere else).

If someone else is running a server on the specific port, the output will look like this:

install_fig_1

In which case, you can provide another port which you want to connect on. Keep trying ports until you find one free 😉 The port number requested needs to be >1024, as numbers <1024 require root access. To change the port, simply specify the new port against the IP mask:

  1. ./digits-server -b 0.0.0.0:34449

When its working it looks like this:

install_fig_2_working

Then to access it, simply go to the computer’s ip address with the associated port and voila!

Keep in mind, that each user will have their own instance, which is running on a unique port. This article doesn’t discuss how to setup authentication, so anyone can submit jobs. If your setup is behind a VPN (as ours is), this isn’t so terrible, but users need to be aware that they’re accessing the correct instance!

Troubleshooting

I had one small problem, where I received the error:

gdk_cursor_new_for_display: assertion 'GDK_IS_DISPLAY (display)' failed

while starting the server.  As I mentioned in the issue, I was able to work around this easily by using a virtual frame buffer with the following command:

  1. xvfb-run ./digits-server

 

Final Thoughts

The only slightly annoying issue is that Digits requires LMDBs instead of leveldbs. They said that they dropped leveldb support since it isn’t supported cross platform. This in itself is fine, except that all of the database I currently have are in leveldb format, so if I want to use them in the future i need to remake them using LMDB. In case anyone is interested, I opt for leveldbs since the file sizes appear to be significantly smaller in my (and other) use case.

Other than that, Nvidia Digits seems to be providing some very nice functionality, taking me away from writing scripts on the command line and instead having a beautiful UI to work with.

Very excited for the new year and a chance to delve deeper into it!

7 thoughts on “How to Install Nvidia Digits”

  1. ubuntu@ubuntu:~/caffe-nv$ cp ~/caffe/Makefile.config .
    cp: cannot stat ‘/home/ubuntu/caffe/Makefile.config’: No such file or directory

    I dont understand, can you explain this to me. Thank you

    1. note the comment underneath the line. i’ve used a working version i already had available. you’ll need to modify the base example to appropriate values for your particular system

  2. I just want to know how we can download all the charts that we get after successful training. I need to paste all those graphs in my research paper.

    1. wow, sorry, i have no idea…i think the last digits update was in 2017? we don’t use this tool anymore. It sounds like you would have to screenshot them or look in its directory structure for image files, they should be stored somewhere!

Leave a Reply

Your email address will not be published. Required fields are marked *