Find and Restore deleted file on git

If you have ever neglected some legacy code for a while and then gone back later to restore it, the most annoying part is if you deleted a file that you needed in the back end. I was tasked with something similar and found that git saved the day but getting the commands right was a little tricky, so I thought i’d share them.

git log --diff-filter=D --summary | grep delete
git log --all -- FILEPATH
git show <SHA> -- <path-to-file>

The first command shows the paths of the files deleted across all branches in the git repo. Placing the file path in the second command, will give you the commits and SHA for what resulted in the deletion of that  and then using them in the third, will give you the file you need.

 

Hope this helps!!!

Installing Proxmox 3.3 (bare_metal)

The last time I set-up my own proxmox server, its latest version 2.0 had just come out and instead of a bare_metal install, I installed debian and installed proxmox on top of it. The separated installation was rather long and exhausting, and recently a colleague decided to install 3.3 and was having considerable difficulty so I decided to go through the motions again and install from scratch a bare_metal proxmox solution.

So it began with downloading the .iso file. However, direct download was extremely slow so I shifted to the torrent download and it was extremely fast. Took about 5 minutes.

Now came the tricky part. Making this iso into a live USB. Tricky because I am using Ubuntu, and there is no clear cut method to create a Live-USB, and proper resources are hard to find for the actual process. So following are the steps I followed to get a Live-USB with the downloaded proxmox image after plugging in the USB. (Copied mostly from Superuser)

  1.  Run “sudo fdisk -l” and get the device the on which your USB is mounted.(in my case it was mounted on /dev/sdf1)
  2. Unmount the device “unmount /dev/sdf1”
  3. (Optional, but I do it just for precaution) Format the USB to FAT32. “sudo mkdosfs -n ‘USB-Drive-Name’ -I /dev/sdf -F 32”
  4. Rename the .iso to .raw .”mv proxmox-3-3.iso proxmox.raw”.
  5. Move the .raw file to the usb “sudo dd if=proxmox.raw of=/dev/sdf bs=4k”
  6. Sync and eject the usb “sync; sudo eject /dev/sdf “

And voilla!  The Live-USB is prepared.

Now I plugged in my USB into my intended proxmox server, checked BIOS on restart to confirm USB as primary boot, and as soon as the USB loaded, there was a simple point and click interface for the rest of the installation. Navigating through it is extremely simple (unlike before), the network config is mostly automatic, and then restart and remove the USB…

And I am done…Proxmox 3.3 Server installed….

 

Hope this helps…!!!

Bootstrap Django templates and admin: A complete step by step

Bootstrap has made the life of back-end developers a little easier. With the ability to make almost all the apps look cleaner then how they would originally appear, it has indeed been a great tool. However, despite the helpful tutorials and examples out there, as is the case with many, they all suffer from the same three flaws:

  1. No tutorial can be a “one size fits all” solution, and that is a reality. Which is why there are many tutorials out there and the reason why I am writing one as well.
  2. Deprecation with time. All tutorials suffer the same time deprecation, where packages are updated, deprecated or removed, configurations altered etc. Unfortunately, the tutorials mostly stay static and eventually end up being a good start guide but no longer step by step instructions. Though I would like to avoid this eventuality, I do not guarantee it.
  3. Assumption of knowledge. Every tutorial assumes a certain degree of know-how of the technologies involved, which sometimes, is just asking too much but is rarely avoidable, otherwise, a post as simple as this would have to be a book.

Since I am a novice at bootstrap (if it works its beautiful),  I decided, that I would go through the motions of bootstrapping a simple Django application, both the front-end template, as well as, the admin section. Depending on how long it would take to get this functional, this post may take more than a few days for me to publish and recognizing the fact that people rarely give details of what OS and what versions of the packages are involved, I will try to be more succinct in my explanation.

Knowledge assumptions:

  • python and Python-Django Framework
  • virtualenv and pip

Operating System and Python:

I am developing on Ubuntu, using Python 2.7.6 . Here are the particulars:

#command to find the release
cat /etc/lsb-release

#Result
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"

Setup:

Setup a virtual environment and a basic Django project, with a dummy app. The following snippet shows all the steps I followed to achieve this.

#command to create virtual env using virtualenvironment wrapper
mkvirtualenv bootstrap_test
#output
'''
New python executable in bootstrap_test/bin/python
Installing setuptools, pip...done.
'''

#activate venv
workon bootstrap_test

#install django
pip install django
#output
'''
Downloading/unpacking django
  Downloading Django-1.7-py2.py3-none-any.whl (7.4MB): 7.4MB downloaded
Installing collected packages: django
Successfully installed django
Cleaning up...
'''

#checking all packages
pip freeze
#output
'''
Django==1.7
argparse==1.2.1
wsgiref==0.1.2
'''

#create django project
django-admin.py startproject bstrap

#create dummy app inside project
django-admin.py startapp dummy

#checking bstrap project directory structure using tree
tree bstrap
#output
'''
bstrap
├── bstrap
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── dummy
│   ├── admin.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
└── manage.py
'''

The above steps are pretty simplistic and if you are confused at this point, continuing might be a little hard. Before continuing to the next phase, go to ‘bstrap/bstrap/settings.py‘ and add dummy to the installed apps, so it looks something like:

# Application definition
INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'dummy',
)

Now before we delve into bootstrapping, lets just check that the admin section of the app is working, obviously, first we run migrate, and set the admin user and password. Then do a runserver, and assuming that you did not specify an ip, the server will be launched at http://127.0.0.1:8000/.

Since we haven’t configured any views, only the admin site will be working at the moment, and that is all we need.

Navigate to http://127.0.0.1:8000/admin/ and you will see the standard django admin template after you login with the superuser you created during migration.

After ensuring that the app works; we begin with the simple bootstrap objective first…bootstrap django admin.

Django Admin Bootstrap:

The major reference for this particular section comes from Riccardo Forina Blog , who designed and implemented the pip package we will be using. For all the details as to how to achieve this, please visit the original blog post. Here I will be repeating what he did and showing the output of the work.  As per his instructions:

  1. install the pip package django-admin-bootstrapped in your virtual environment by running “pip install django-admin-bootstrapped”
  2. Add ‘django_admin_bootstrapped.bootstrap3‘, into the INSTALLED_APPS before 'django.contrib.admin'.

I followed the above two steps, re-ran my runserver, navigated to http://127.0.0.1:8000/admin/ and voilla!!!  the admin was bootstrapped….

 

 Django Templates Bootstrap:

So first, lets add the static configuration to the setting file. I have a configuration that I use in most of my application, which is as follows:

#BASE_DIR = os.path.dirname(os.path.dirname(__file__))
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.6/howto/static-files/

STATIC_URL = '/static/'
# STATIC_PATH = os.path.join(BASE_DIR,'static')
# STATIC_ROOT_URL = ''
STATIC_ROOT = os.path.join(BASE_DIR,'static_media')

STATICFILES_DIRS = (
  os.path.join(BASE_DIR, 'static/'),
)

#templates
TEMPLATE_PATH = os.path.join(BASE_DIR, 'templates')

TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
    TEMPLATE_PATH,
)

Next, create a folder in your main project directory called “static”, download Bootstrap and add the unzipped files to the “static” folder you created, so that your project folder structure now looks something like this:

bstrap/
├── bstrap
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── settings.py
│   ├── settings.pyc
│   ├── urls.py
│   ├── urls.pyc
│   ├── wsgi.py
│   └── wsgi.pyc
├── db.sqlite3
├── dummy
│   ├── admin.py
│   ├── admin.pyc
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── migrations
│   │   ├── __init__.py
│   │   └── __init__.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── tests.py
│   └── views.py
├── manage.py
└── static
    └── bootstrap
        ├── css
        │   ├── bootstrap.css
        │   ├── bootstrap.css.map
        │   ├── bootstrap.min.css
        │   ├── bootstrap-theme.css
        │   ├── bootstrap-theme.css.map
        │   └── bootstrap-theme.min.css
        ├── fonts
        │   ├── glyphicons-halflings-regular.eot
        │   ├── glyphicons-halflings-regular.svg
        │   ├── glyphicons-halflings-regular.ttf
        │   └── glyphicons-halflings-regular.woff
        └── js
            ├── bootstrap.js
            └── bootstrap.min.js

Now you have setup the basis for bootstrapping your templates. I created a templates folder and added an example template I copied from the bootstrap website.(Starter Template). I called it base.html.

Now in order to actually render and view the bootstrap result, I changed two files. The urls in the bstrap app to look like…

from django.conf.urls import patterns, include, url
from django.contrib import admin

urlpatterns = patterns('',
    # Examples:
     url(r'^$', 'dummy.views.home', name='home'),
    # url(r'^blog/', include('blog.urls')),

    url(r'^admin/', include(admin.site.urls)),
)

and the views in the dummy app to look like…

from django.shortcuts import render
from django.shortcuts import render_to_response

def home(request):
    return render_to_response('base.html')

And you are done! Running the runserver now, and navigating to http://127.0.0.1:8000/ should show you the “Starter Bootstrap template”…

 

Source Code:

https://bitbucket.org/ssaqibyawargmailcom/bstrap/

 

P.S if you are looking for an easy way to boostrap your forms in django, I recommend you navigate to django-bootstrap-form app and use that.

 

Hope this helps!!!

The hidden dependencies of Python packages

Ever since I started using pip and python  on Ubuntu, I have always had certain packages that would be silent about what dependencies they require in order to work, and only when they fail to install, is when you figure that out. I was just installing another virtual environment just now, and decided that I should probably put up a post of any such packages that I come across.

So lets begin with the one that had me searching for its dependencies just now, which would be lxml==3.3.3.

Here is the apt command with all the packages that I had to install.

sudo apt-get install libxml2 libxslt1.1 libxml2-dev libxslt1-dev python-libxml2 python-libxslt1 python-dev python-setuptools zlib1g-dev

Hope this helps!!!

P.S. This article is more like a reminder to self of the dependencies so that I do not need to go searching for them again…coming up next is psycopg2

Performing SNMP walk and get operations using SNMP4j and Java

I was recently tasked with writing some java code to retrieve values, using SNMP and Java. Obviously, I turned to  the SNMP4j library, but to my dismay, the documentation was a little vague on how to simply achieve a GET and WALK operation, and I did not want to spend that much time writing code that was going to be used only once.

Thankfully Google provided some good examples, but as it happens often, none of them were quite complete, as to say, not exactly what I needed. So I am sharing my scripts that I used to achieve my objectives.

SNMP Get

import org.snmp4j.CommunityTarget;
import org.snmp4j.PDU;
import org.snmp4j.Snmp;
import org.snmp4j.TransportMapping;
import org.snmp4j.event.ResponseEvent;
import org.snmp4j.event.ResponseListener;
import org.snmp4j.mp.SnmpConstants;
import org.snmp4j.smi.*;
import org.snmp4j.transport.DefaultUdpTransportMapping;

public class SnmpGet {

    public SnmpGet() {
		// TODO Auto-generated constructor stub
	}


	/*
	 *  SnmpGet method

	 * return Response for given OID from the Device.

	 */

	public static String snmpGet(String strAddress, String community, String strOID, int snmpVersion)

	{

		String str="";

		try

		{

			OctetString community1 = new OctetString(community);

			strAddress= strAddress+"/" + 161;

			Address targetaddress = new UdpAddress(strAddress);

			TransportMapping transport = new DefaultUdpTransportMapping();

			transport.listen();

			CommunityTarget comtarget = new CommunityTarget();

			comtarget.setCommunity(community1);

			if(snmpVersion == 1){
				comtarget.setVersion(SnmpConstants.version1);
			}
			
			if(snmpVersion == 2){
				comtarget.setVersion(SnmpConstants.version2c);
			}
			
			if(snmpVersion == 3){
				comtarget.setVersion(SnmpConstants.version3);
			}
	

			comtarget.setAddress(targetaddress);

			comtarget.setRetries(2);

			comtarget.setTimeout(5000);

			PDU pdu = new PDU();

			ResponseEvent response;

			Snmp snmp;

			pdu.add(new VariableBinding(new OID(strOID)));

			pdu.setType(PDU.GET);

			snmp = new Snmp(transport);

			response = snmp.get(pdu,comtarget);

			if(response != null)

			{

				if(response.getResponse().getErrorStatusText().equalsIgnoreCase("Success"))

				{

					PDU pduresponse=response.getResponse();

					str=pduresponse.getVariableBindings().firstElement().toString();

					if(str.contains("="))

					{

						int len = str.indexOf("=");

						str=str.substring(len+1, str.length());

					}

				}

			}

			else

			{

				System.out.println("Feeling like a TimeOut occured ");

			}

			snmp.close();

		} catch(Exception e) { e.printStackTrace(); }

		System.out.println("Response="+str);

		return str;

	}
	
	
}

SNMP Walk

import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
 
import org.snmp4j.CommunityTarget;
import org.snmp4j.Snmp;
import org.snmp4j.TransportMapping;
import org.snmp4j.mp.SnmpConstants;
import org.snmp4j.smi.Address;
import org.snmp4j.smi.GenericAddress;
import org.snmp4j.smi.OID;
import org.snmp4j.smi.OctetString;
import org.snmp4j.smi.VariableBinding;
import org.snmp4j.transport.DefaultUdpTransportMapping;
import org.snmp4j.util.DefaultPDUFactory;
import org.snmp4j.util.TreeUtils;
import org.snmp4j.util.TreeEvent;
 
public class Snmpwalk {  
  private String targetAddr;
  private String oidStr;
  private String commStr;
  private int snmpVersion;
  private String portNum;  
  private String usage;
  private ArrayList<String> results = new ArrayList<String>();
    
  Snmpwalk() throws IOException{
    // Set default value.
    targetAddr = null;
    oidStr = null;
    commStr = "public";
    snmpVersion = SnmpConstants.version2c;
    portNum =  "161";
    usage = "Usage: snmpwalk [-c communityName -p portNumber -v snmpVersion(1 or 2)] targetAddr oid";
  }
  
  public Snmpwalk(HashMap<String, String> values){
      targetAddr = values.get("ipaddr");
	  oidStr = values.get("oid");
	  commStr = values.get("communityString");
	  int tempVersion = Integer.parseInt(values.get("version"));
	  if(tempVersion == 1){
		  snmpVersion = SnmpConstants.version1;
	  }
	  if(tempVersion == 2){
		  snmpVersion = SnmpConstants.version2c;
	  }
	  if(tempVersion == 3){
		  snmpVersion = SnmpConstants.version3;
	  }
	  portNum =  "161";
  }
  
  public void doSnmpwalk() throws IOException  {
    Address targetAddress = GenericAddress.parse("udp:"+ targetAddr + "/" + portNum);
    TransportMapping transport = new DefaultUdpTransportMapping();
    Snmp snmp = new Snmp(transport);
    transport.listen();
  
    // setting up target
    CommunityTarget target = new CommunityTarget();
    target.setCommunity(new OctetString(commStr));
    target.setAddress(targetAddress);
    target.setRetries(3);
    target.setTimeout(1000 * 3);
    target.setVersion(snmpVersion);
  
    OID oid = null;
    try{
      oid = new OID(oidStr);
    }
    catch(RuntimeException ex){
      System.out.println("OID is not specified correctly.");
      System.exit(1);
    }
  
    TreeUtils treeUtils = new TreeUtils(snmp, new DefaultPDUFactory());      
    List<TreeEvent> events = treeUtils.getSubtree(target, oid);
    if(events == null || events.size() == 0){
      System.out.println("No result returned.");
      System.exit(1);
    }
    
    // Get snmpwalk result.
    for (TreeEvent event : events) {
      if(event != null){
        if (event.isError()) {
            System.err.println("oid [" + oid + "] " + event.getErrorMessage());
          }
            
        VariableBinding[] varBindings = event.getVariableBindings();
        if(varBindings == null || varBindings.length == 0){
          System.out.println("No result returned.");
        }
        for (VariableBinding varBinding : varBindings) {
          results.add(varBinding.getVariable().toString());
        }
      }
    }
    
    //print results
    for(String value: results){
    	System.out.println(value);
    }
    snmp.close();
  }
  
  public boolean isInResults(String value){
	  return results.contains(value);
  }
    
  
  private void checkAndSetArgs(String[] args){
    if(args.length < 2){
      System.err.println(usage);
      System.exit(1);
    }
    
    for (int i=0; i<args.length; i++){
      if("-c".equals(args[i])){
        commStr = args[++i]; 
      }
      else if ("-v".equals(args[i])){
        if(Integer.parseInt(args[++i]) == 1){
          snmpVersion = SnmpConstants.version1;
        }
        else {
          snmpVersion = SnmpConstants.version2c;
        }
      }
      else if ("-p".equals(args[i])){
        portNum = args[++i];
      }
      else{
        targetAddr = args[i++];
        oidStr = args[i];
      }
    }
    if(targetAddr == null || oidStr == null){
      System.err.println(usage);
      System.exit(1);
    }
  }

The above code has been tested to work with SNMP version 1 and 2, but should work with 3 as well.

Hope this helps…!