Author: harry

Security actions on unprotected SSH Keys (Staff/PhD)

harry

Please follow this guide to encrypt your SSH private keys

 

Self-managed devices

Users on self-managed devices are strongly advised to review the security of their SSH keys and remove them if the are not protected with a passphrase.

You will know if your keys are protected with a passphrase if your get prompted for it when you try to connect to ‘frank’ or any other Compute Server.

Bellow you will find some guidelines to find out if you have unprotected SSH keys for the 3 main Operating Systems:

SSH Private keys generated using Putty or MobaXterm have the extension .ppk. Search your Windows device for files with such extension.

Open the file with Notepad (or any text editor).

Secure Private SSH Key

A secure private key protected with a passphrase has something like that at the top (attention at the ‘Encryption‘ line) :

PuTTY-User-Key-File-2: ssh-rsa
Encryption: aes256-cbc
Comment: rsa-key-20171114
Public-Lines: 6

The ‘Encryption‘ line shows the encryption algorithm used to protect your private key when you generated.

Unprotected Private SSH Key

An unprotected private key has something like that at the top (attention at the ‘Encryption‘ line) :

PuTTY-User-Key-File-2: ssh-rsa
Encryption: none
Comment: rsa-key-20200506
Public-Lines: 6

If your private key mentions there is no Encryption, as above, DELETE the private and public keys immediately and generate a new one, following our guide here: http://support.eecs.qmul.ac.uk/services/ssh/

On your self-managed Linux device, open a Terminal and run the following command on your Private SSH key:

$ ssh-keygen -y -P "" -f <PATH_TO_PRIVATE_SSH_KEY>

Secure Private SSH Key

If the output of the command is incorrect passphrase supplied to decrypt private key , it means your key is already encrypted. For example:

$ ssh-keygen -y -P "" -f /home/harry/.ssh/id_rsa_encrypted
Load key "/home/harry/.ssh/id_rsa)_encrypted": incorrect passphrase supplied to decrypt private key

Unprotected Private SSH Key

If the output is the Public Key, associated to that particular private key, it means your key is not protected, for .e.g:

$ ssh-keygen -y -P "" -f /home/harry/.ssh/id_rsa_no_encryption
ssh-rsa AAFFGFFFAAB3NzaC1yc2EAAAADAQABAAABAQDEzlgGMuIV1dRHo5E7CJbOF8QHMz2G/ndqP8GwKzmqS5jKiwghAKprp1vB2Q5jcHIN7/ycOEYQw4HzvHqKd2BpygArQCiMqnkgHVRogzJEUIuQ0qNAe2ao+krCJz12Ihz

 

On your self-managed MacOS device, open a Terminal and run the following command on your Private SSH key:

$ ssh-keygen -y -P "" -f <PATH_TO_PRIVATE_SSH_KEY>

Secure Private SSH Key

If the output of the command is incorrect passphrase supplied to decrypt private key , it means your key is already encrypted. For example:

$ ssh-keygen -y -P "" -f /home/harry/.ssh/id_rsa_encrypted
Load key "/home/harry/.ssh/id_rsa)_encrypted": incorrect passphrase supplied to decrypt private key

Unprotected Private SSH Key

If the output is the Public Key, associated to that particular private key, it means your key is not protected, for .e.g:

$ ssh-keygen -y -P "" -f /home/harry/.ssh/id_rsa_no_encryption
ssh-rsa AAFFGFFFAAB3NzaC1yc2EAAAADAQABAAABAQDEzlgGMuIV1dRHo5E7CJbOF8QHMz2G/ndqP8GwKzmqS5jKiwghAKprp1vB2Q5jcHIN7/ycOEYQw4HzvHqKd2BpygArQCiMqnkgHVRogzJEUIuQ0qNAe2ao+krCJz12Ihz

 

 

 

 

Anaconda

harry

Anaconda is the world’s most popular Python/R data science and machine learning platform. It contains more than 7,500+ Python/R data science packages, it has become industry standard platform for developing, testing Python/R applications and for Deep Learning.

Versions

All Anaconda versions are available as environment modules. We provide only Anaconda with Python 3 support since Python 2 reached its End of Life and the latest Machine Learning tools have minimal or no support for Python 2.

We recommend using the Anaconda we provide as environment module, over custom installations in your home directory, cause it offers more control on your local environment.

Many users that have built anaconda manually at their home directories, have reported major issues affecting their EECS Desktop Environment

 

Usage

 

Loading the module

To use the default (latest) version of Anaconda:

$ module load anaconda3

Check the versions

$ conda --version
conda 4.7.12

$ python -V
Python 3.7.4

Create a new environment

Create a new Conda environment with a custom name after the -n option:

$ conda create --quiet --yes -n mycondaenv

List conda environments

$ conda env list
# conda environments:
#
mycondaenv /homes/$USER/.conda/envs/mycondaenv
base * /import/linux/anaconda/3/2019.10

Your new environments is listed and also note the * at the ‘base’ environment, it indicated the current active environment.

Activate the environment

To activate your conda environment and start installing packages:

$ source activate mycondaenv
(mycondaenv) $

Install packages

To install a package with its dependencies in your conda environment:

(mycondaenv) $ conda install <package_name>

List installed packages

To list the available  packages in your current activated conda environment:

(mycondaenv) $ conda list

Deactivate the environment

To stop using the conda environment and exit to your regular shell:

(mycondaenv) $ source deactivate

 

References

Python

harry
Requirements:

  • Environment Modules

The guide assumes basic understanding of the above requirements

Python is a general-purpose, versatile and popular programming language. It’s great as a first language because it is concise and easy to read, and it is also a good language to have in any programmer’s stack as it can be used for everything from web development to software development and scientific applications.

 

Versions

The following versions are available on EECS Servers and Desktops:

Version Available as
 Binaries PATH
2.7 Default system version /bin/python
3.4 RPM package, Environment module /bin/python3.4
3.6.9 Environment module /import/linux/python/3.6.9/bin/python
3.6.10 Environment module /import/linux/python/3.6.10/bin/python
3.7.7 Environment module /import/linux/python/3.7.7/bin/python
3.8.2 Environment module /import/linux/python/3.8.2/bin/python

Using Python

The versions in the table above are available as environment modules, that enables users to load and switch between different Python versions with just one command.

For example, if you need to use the provided Python 3.6.10 , from the command line:

$ module load python/3.6.10

To check the version of the Python you are now using:

$ python -V
Python 3.6.10

Installing Python packages

Users do not have “admin” priviliges to install Python packages on the EECS systems, but any Python packages can be installed inside a user’s home directory with more than one ways. We recommend using virtualenv for more efficient workflow.

Choose any tab to how you can install Python packages in your home directory:

 

Virtualenv give the ability to create isolated Python Virtual Environments with specific packages and different versions for every project you are working on. Python Virtualenv helps keeping  the unique dependencies for every project separate, locally and in an isolated folder with the same name of the virtualenv.

To work with a specific  virtualenv, you must “activate” and when you’ve finished you have to “deactivate” it.

Load the module

First, load the Python versions of your choice from the available environment modules . For example, to use Python 3.6.10:

  $ module load python/3.6.10

Check that you loaded the right Python version:

$ python -V 
Python 3.6.10

Create the virtualenv

To create a virtualenv:

$ python -m venv <venv_name>

That will create a folder with the name you provided above which contains your new Python virtualenv.

$ ls <venv_name>/
bin include lib lib64 pyvenv.cfg

Activate the virtualenv

To activate and start using the new virtualenv:

$ source <venv_name>/bin/activate
(<venv_name>) $

You will notice that your prompt has changed and the name of the activated virtualenv has been added at the beginning of each line. That shows that you have successfully activated the virtualenv and its name.

Now you can start installing your own packages.

Install a package

To install a Python package, simply run:

(<venv_name>) $ pip install <package_name>

To install a specific version of a package you must define it like:

(<venv_name>) $ pip install <package_name>==<version>

Deactivate virtualenv

To stop using the current virtualenv:

(<venv_name>) $ deactivate
$

Anaconda is an open-source distribution of Python and R for scientific computing. It provides an easy way to manage the Python packages you install and the virtualenv you create.

For more info on how to install Python packages using Anaconda, read our Anaconda Guide.

Python packages can be installed to your $HOME directory by adding the --user option. This requires to manually update your local SHELL to point at the location of the installed binaries and libraries.

Load the module

First, load the Python versions of your choice from the available environment modules . For example, to use Python 3.6.10:

  $ module load python/3.6.10

Install a package

To install a Python package,  run:

$ pip install --user <package_name>

It will install the contents of the package at:

• Binaries: $HOME/.local/bin

• Modules/Libraries: $HOME/.local/lib/pythonX.Y/site-packages

Update local environment

Depending on your local environment variables, you might need to add at least the following (replace X.Y with the Python version you used to install the packages):

$ export PATH=~/.local/bin:$PATH
$ export PYTHONPATH=​$HOME/.local/lib/pythonX.Y/site-packages:$PYTHONPATH

References

 

Tensorflow

harry

Requirements

Make sure you have enough available disk space in your home directory: Disk quotas

This guide assumes you are already familiar with:

Info

TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications. For information and usage, please read the official documentation: Learn Tensorflow

TensorFlow 1.x uses separate packages for CPU and GPU support, tensorflow (non-GPU dependent) and tensorflow-gpu(requires CUDA/cuDNN).

TensorFlow 2.x is released with both CPU and GPU support.

Install tensorflow

This example uses Anaconda to install tensorflow-gpu==1.15 inside a Python 3.6.10 virtual environment with the name dev_tensorflow_1, using CUDA-10.0 with cuDNN-7.5.9

Click on any tab to find how to install tensorflow using different methods.

Loading the module

Load any of the available Python versions we provide, in the form of environment modules . For example, to use the available Python 3.6.10:

  $ module load python/3.6.10

Check the Python version:

$ python -V 
Python 3.6.10

Loading CUDA

Load the combination of CUDA/cuDNN that is suitable for you. For example:

 $ module load cuda/10.0-cudnn7.5.0

Create the environment

Create a new Python Virtual environment (venv) and define a name, for example to create a venv with the name dev_tensorflow_1:

$ python -m venv dev_tensorflow_1

Activate the environment:

 $ source dev_tensorflow_1/bin/activate
(dev_tensorflow_1) $

Install packages

Install the tensorflow version you need:

(dev_tensorflow_1) $ pip install tensorflow-gpu==1.15

To check the version of tensorflow you just installed:

(dev_tensorflow_1) $ python
>>> import tensorflow as tf
>>> print(tf.VERSION)
1.15.0

 

Loading the module

To load the default installed version of Anaconda (Python3), load the anaconda3 module:

$ module load anaconda3

Check the Python version:

$ python -V 
Python 3.7.4

Loading CUDA

Load the combination of CUDA/cuDNN that is suitable for you. For example:

 $ module load cuda/10.0-cudnn7.5.0

Create the environment

Create a new Anaconda environment and define a name, for example to create an environment with the name dev_tensorflow_1:

$ conda create --yes --quiet --name dev_tensorflow_1

Activate the environment:

$ source activate dev_tensorflow_1
(dev_tensorflow_1) $

Install packages

Install the tensorflow version you need:

(dev_tensorflow_1) $ conda install tensorflow==1.15

Check the installed packages:

(dev_tensorflow_1) $ conda list

Check the installed tensorflow version

(dev_tensorflow_1) $ python -c "import tensorflow as tf; print(tf.VERSION)"
1.15.0

 

 

References

Anaconda and Miniconda for Research

harry

The latest versions of Miniconda 3 and Anaconda 3 are now available to all EECS Research Servers and Managed Linux Desktops, as environment modules.

We have been recommending against the usage of anaconda in our infrastructure for quite a while, since it was breaking the user’s desktop environments on the desktops when it was used incorrectly, making it hard for the users to recover from that bad state. Using Anaconda and Miniconda as environment modules, gives more control to the user on how, when and where anaconda environments will be initialised, removes the need to initialise the user’s shell with anaconda and removes the complexity of manually modifying your SHELL via the .bashrc file.

Read More

Security Update: SMB/CIFS disabled for unmanaged devices

harry

Due to the recent major incident with the  RYUK Ransomware attack at the School of SEMS, we have DISABLED access to unmanaged devices using the SMB/CIFS protocol to the networks shares on the staff login-server ‘frank.eecs.qmul.ac.uk’.

This change will not affect the EECS Managed Desktops, which will keep using the networks shares via ‘tofu’ using the SMB/CIFS protocol. Also, printing from unmanaged devices (which requires SMB connection to frank.eecs.qmul.ac.uk) will not be affected.

Read More