Installing and Troubleshooting Netbox

Today because i had some free time, i start to install and configure Netbox to a new virtual machine with Ubuntu 18.04. Found some tutorials online but looks like this was not an easy concept like having a short apt-get install netbox. This is not the case here.

One of the best and very recently looks like to be this one. Credit to the author.
Installation was pretty fast.

Issues:

Make sure you put ” inside [] for the IP’s. Example:
ALLOWED_HOSTS = ['10.10.10.12']
Because at point 1 i started only with = [] at the end of tutorial i had a white screen error with “Bad Request (400)”.

Checking the status of the supervisor i can confirm that here i have some problems. Actually there is a loop running, cpu of the VM was getting crazy.

root@netbox:/# systemctl status supervisor
● supervisor.service - Supervisor process control system for UNIX
Loaded: loaded (/lib/systemd/system/supervisor.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-07-27 11:46:11 UTC; 4s ago
Docs: http://supervisord.org
Process: 12810 ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown (code=exited, status=0/SUCCESS)
Main PID: 15679 (supervisord)
Tasks: 6 (limit: 4660)
CGroup: /system.slice/supervisor.service
├─15679 /usr/bin/python /usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf
├─15699 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
├─15700 python3 /opt/netbox/netbox/manage.py rqworker
├─15705 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
├─15706 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
└─15707 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
Jul 27 11:46:11 netbox systemd[1]: Started Supervisor process control system for UNIX.
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,105 CRIT Supervisor running as root (no user in config file)
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,106 INFO Included extra file "/etc/supervisor/conf.d/netbox.conf" during parsing
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,113 INFO RPC interface 'supervisor' initialized
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,113 CRIT Server 'unix_http_server' running without any HTTP authentication checking
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,113 INFO supervisord started with pid 15679
Jul 27 11:46:13 netbox supervisord[15679]: 2020-07-27 11:46:13,115 INFO spawned: 'netbox' with pid 15699
Jul 27 11:46:13 netbox supervisord[15679]: 2020-07-27 11:46:13,117 INFO spawned: 'netbox-rqworker' with pid 15700
Jul 27 11:46:14 netbox supervisord[15679]: 2020-07-27 11:46:14,379 INFO success: netbox entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Jul 27 11:46:14 netbox supervisord[15679]: 2020-07-27 11:46:14,379 INFO success: netbox-rqworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

3. I disabled supervisor with with comand below and i start to check what is going on.

root@netbox:/# systemctl stop supervisor

I checked the actually is supervisor doing. Looks like is running multiple command and i started to run them manually.

paulierco@netbox:~$ cat /etc/supervisor/conf.d/netbox.conf
[program:netbox]
command = gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
directory = /opt/netbox/netbox/
user = www-data
[program:netbox-rqworker]
command = python3 /opt/netbox/netbox/manage.py rqworker
directory = /opt/netbox/netbox/
user = www-data

I discovered that if i run the command below, netbox start to run without any problems. CPU was also fine, i would say mostly idle.

gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi

When i checked the other command, i saw that i start to have problems, with a error at the end.

python3 /opt/netbox/netbox/manage.py rqworker

Error ends with:

TypeError: hset() got an unexpected keyword argument 'mapping'

I was thinking that this issue might be because of the python3 version i had installed. It was something like python3.6. You can check this with python3 –version.
Following this tutorial i upgrade the Ubuntu 18.04 to python 3.8.5 and now i have:

paulierco@netbox:~$ python3 --version
Python 3.8.5

Because i had a new version of python3, i had to reinstall all the requirements using tutorial (step 5 from here):

sudo su - root 
pip3 install -r /opt/netbox/requirements.txt 
pip3 install gunicorn exit

Then i start to have other issues with apt but somehow the issue was still there:

root@netbox:~# python3 /opt/netbox/netbox/manage.py rqworker
/opt/netbox/netbox/utilities/api.py:103: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if obj is '':
/opt/netbox/netbox/utilities/api.py:118: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if data is '':
Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/opt/netbox/netbox/extras/management/commands/rqworker.py", line 16, in handle
    super().handle(*args, **options)
  File "/usr/local/lib/python3.8/dist-packages/django_rq/management/commands/rqworker.py", line 102, in handle
    w.work(burst=options.get('burst', False), with_scheduler=options.get('with_scheduler', False), logging_level=level)
  File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 511, in work
    self.register_birth()
  File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 296, in register_birth
    p.hset(key, mapping=mapping)
TypeError: hset() got an unexpected keyword argument 'mapping'

Then i saw somewhere that this might be related to redis but the application related to redis-py and then i saw that my apt was damaged.

root@netbox:~# pip install -e git://github.com/andymccurdy/redis-py.git#egg=master
Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 28, in <module>
    from CommandNotFound import CommandNotFound
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 19, in <module>
    from CommandNotFound.db.db import SqliteDatabase
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 5, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
root@netbox:~# redis-py
Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 28, in <module>
    from CommandNotFound import CommandNotFound
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 19, in <module>
    from CommandNotFound.db.db import SqliteDatabase
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 5, in <module>
    import apt_pkg

To fix this had to:

Go to /usr/lib/python3/dist-packages#
do a ls -als

root@netbox:/usr/lib/python3/dist-packages# ls -als
total 1872
 12 drwxr-xr-x 105 root root  12288 Jul 27 11:42 .
  4 drwxr-xr-x   3 root root   4096 Feb  3 18:22 ..
  4 drwxr-xr-x   2 root root   4096 Feb  3 18:23 Automat-0.6.0.egg-info
  4 drwxr-xr-x   4 root root   4096 Feb  3 18:23 CommandNotFound
  4 drwxr-xr-x  11 root root   4096 Jul 27 09:23 Crypto
  4 drwxr-xr-x   3 root root   4096 Feb  3 18:24 DistUpgrade
  4 drwxr-xr-x   3 root root   4096 Jul 27 09:18 HweSupportStatus
  4 drwxr-xr-x   2 root root   4096 Feb  3 18:23 Jinja2-2.10.egg-info
  4 drwxr-xr-x   3 root root   4096 Feb  3 18:24 LanguageSelector
  4 drwxr-xr-x   2 root root   4096 Feb  3 18:23 MarkupSafe-1.0.egg-info
  4 drwxr-xr-x   3 root root   4096 Feb  3 18:23 OpenSSL
  4 -rw-r--r--   1 root root    193 Aug  4  2017 PAM-0.4.2.egg-info
 20 -rw-r--r--   1 root root  19888 Aug  4  2017 PAM.cpython-36m-x86_64-linux-gnu.so
  4 drwxr-xr-x   2 root root   4096 Feb  3 18:23 PyJWT-1.5.3.egg-info
  4 -rw-r--r--   1 root root   1515 Aug  5  2017 PyYAML-3.12.egg-info
  4 -rw-r--r--   1 root root    119 Jul 17 12:50 README.txt
  4 drwxr-xr-x   2 root root   4096 Jul 27 09:23 SecretStorage-2.3.1.egg-info
  4 drwxr-xr-x   2 root root   4096 Jul 27 09:14 Twisted-17.9.0.egg-info
  4 drwxr-xr-x   5 root root   4096 Jul 27 09:18 UpdateManager
  4 drwxr-xr-x   2 root root   4096 Jul 27 11:33 __pycache__
168 -rw-r--r--   1 root root 169536 Feb 27  2018 _cffi_backend.cpython-36m-x86_64-linux-gnu.so
156 -rw-r--r--   1 root root 159408 Jan 29  2018 _dbus_bindings.cpython-36m-x86_64-linux-gnu.so
 20 -rw-r--r--   1 root root  19144 Jan 29  2018 _dbus_glib_bindings.cpython-36m-x86_64-linux-gnu.so
 44 -rw-r--r--   1 root root  43616 Jan 12  2018 _snack.cpython-36m-x86_64-linux-gnu.so
  4 -rw-r--r--   1 root root     21 Aug 26  2014 _version.py
224 -rw-r--r--   1 root root 225440 Aug  5  2017 _yaml.cpython-36m-x86_64-linux-gnu.so
  4 drwxr-xr-x   4 root root   4096 Jul 27 09:18 apport
  8 -rw-r--r--   1 root root   8063 May 13 22:12 apport_python_hook.py
  4 drwxr-xr-x   4 root root   4096 Jul 27 11:39 apt
 52 -rw-r--r--   1 root root  51160 Jan 24  2020 apt_inst.cpython-36m-x86_64-linux-gnu.so
  4 -rw-r--r--   1 root root    227 Jan 24  2020 apt_inst.pyi
340 -rw-r--r--   1 root root 346784 Jan 24  2020 apt_pkg.cpython-36m-x86_64-linux-gnu.so
  4 lrwxrwxrwx   1 root root     69 Jul 27 11:42 apt_pkg.cpython-37-x86_64-linux-gnu.so -> /usr/lib/python3/dist-packages/apt_pkg.cpython-38-x86_64-linux-gnu.so
 12 -rw-r--r--   1 root root   8900 Jan 24  2020 apt_pkg.pyi

Next command was:

root@netbox:/usr/lib/python3/dist-packages# cp apt_pkg.cpython-36m-x86_64-linux-gnu.so apt_pkg.so
root@netbox:/usr/lib/python3/dist-packages# apt-get update
Hit:1 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease
Hit:2 http://ro.archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://ro.archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:4 http://ro.archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:5 http://ro.archive.ubuntu.com/ubuntu bionic-security InRelease
Reading package lists... Done

Now as my apt was fixed, back to our supervisor.
To upgrade redis-py we have to reuse the command:

root@netbox:/# pip3 install -e git://github.com/andymccurdy/redis-py.git#egg=master
Obtaining master from git+git://github.com/andymccurdy/redis-py.git#egg=master
  Cloning git://github.com/andymccurdy/redis-py.git to /src/master
  Running setup.py (path:/src/master/setup.py) egg_info for package master produced metadata for project name redis. Fix your #egg=master fragments.
Installing collected packages: redis
  Found existing installation: redis 3.4.1
    Uninstalling redis-3.4.1:
      Successfully uninstalled redis-3.4.1
  Running setup.py develop for redis
Successfully installed redis

Running the last command that had errors i had no error now. This looks fixed.

python3 /opt/netbox/netbox/manage.py rqworker

Back to our application this looks working well.

root@netbox:/# systemctl restart  supervisor
root@netbox:/# systemctl status  supervisor
● supervisor.service - Supervisor process control system for UNIX
   Loaded: loaded (/lib/systemd/system/supervisor.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-07-27 11:46:11 UTC; 4s ago
     Docs: http://supervisord.org
  Process: 12810 ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown (code=exited, status=0/SUCCESS)
 Main PID: 15679 (supervisord)
    Tasks: 6 (limit: 4660)
   CGroup: /system.slice/supervisor.service
           ├─15679 /usr/bin/python /usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf
           ├─15699 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
           ├─15700 python3 /opt/netbox/netbox/manage.py rqworker
           ├─15705 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
           ├─15706 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
           └─15707 /usr/bin/python3 /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi

Jul 27 11:46:11 netbox systemd[1]: Started Supervisor process control system for UNIX.
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,105 CRIT Supervisor running as root (no user in config file)
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,106 INFO Included extra file "/etc/supervisor/conf.d/netbox.conf" during parsing
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,113 INFO RPC interface 'supervisor' initialized
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,113 CRIT Server 'unix_http_server' running without any HTTP authentication checking
Jul 27 11:46:12 netbox supervisord[15679]: 2020-07-27 11:46:12,113 INFO supervisord started with pid 15679
Jul 27 11:46:13 netbox supervisord[15679]: 2020-07-27 11:46:13,115 INFO spawned: 'netbox' with pid 15699
Jul 27 11:46:13 netbox supervisord[15679]: 2020-07-27 11:46:13,117 INFO spawned: 'netbox-rqworker' with pid 15700
Jul 27 11:46:14 netbox supervisord[15679]: 2020-07-27 11:46:14,379 INFO success: netbox entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Jul 27 11:46:14 netbox supervisord[15679]: 2020-07-27 11:46:14,379 INFO success: netbox-rqworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)