Simple 2

I've just about finished the next version of Simple, the markdown based blog that powers this site.

When I first made Simple it was because I disliked WordPress, which seemed a bit too bloated. Then I saw Svbtle and I really liked the minimalist design (mostly the posting interface) and decided to make a clone in Python.

This worked great and powered my blog for three years or so without any issues, but I became a bit tired of the minimal white design and wanted something with a bit more to look at. When I stumbled across the HPSTR Jekyll Theme I really liked the layout and design and decided to adapt that for the next version of Simple.

And so here it is. It tries to stay true to it's name by being simple to use and install whilst still having a decent feature set and a nice design. The editing interface is a styled textarea that grows as you type, and adding an image is a simple as dragging and dropping it onto the page. This will upload the image and insert the right markdown at your cursor position. You can also edit the title by just selecting it and typing.

One thing I really liked about the HPSTR theme is the large image header, and I decided to combine this with the Bing daily image. When writing a post you can view the last 25 daily images by clicking the picture icon in the top right and using the left and right arrows to navigate:

I've tried to make installing Simple a painless as possible. You create a virtual environment for Simple, install the package and then use the 'simple' command to create a blog. Creating and maintaining config files is a pain, so you can use the simple command to create nginx and supervisord config files with the right file paths included.

>> mkdir blog && cd blog
>> python3.4 -m venv env
>> source env/bin/activate && pip install simpleblogging gunicorn
>> simple create
>> nano simple_settings.py
>> simple nginx_config yoursite.com --proxy-port=9009 > /etc/nginx/conf.d/simple.conf
>> simple supervisor_config env/ 9009 >> /etc/supervisord.conf
>> chown -R nobody:nobody ../blog
>> supervisorctl start simple && service nginx reload

And that's all you need.

Exploiting XPath injection vulnerabilities with XCat

I just released XCat 0.7, the companion tool to this paper. XCat is a command line tool to automate the exploitation of Blind XPath Injection Vulnerabilities, utilizing some pretty cool techniques.

The most interesting technique is that xcat can automate out of band attacks to massively speed up extraction of data. In English that means that it can turn a blind injection (where one request equals one bit of data) into a standard injection (where one request can result in many bits of data), essentially making the server send the data to XCat in big chunks. It also comes with a "file shell" option that allows you to access local files on the server through a variety of methods. You can find out how to install it in the documentation here: http://xcat.readthedocs.org/en/latest/, and this post provides a summary of XCat's capabilities.

XPath?

XPath is like SQL for XML. Imagine you had this XML document with a list of users:

<root>
    <user username='Tom' password='pass'/>
    <user username='Jane' password='wyf'/>
    <user username='Steve' password='abcd'/>
</root>

And you wanted to query the existence of a particular user. You could write something like this:

/root/user[@username="Tom"]

That query would return the user node with the attribute 'username' set to 'Tom'. There are lots of better examples on the Wikipedia page if you're interested.

XCat?

Imagine if the query above was part of a form, and the code puts unescaped user input into the username part of the query. If the query finds a result it redirects you somewhere, otherwise it displays an error. An attacker could subvert the query by adding his own logic:

/root/user[@username="Tom" and @password="pass" and "1"="1"]

Now the form will only redirect if the user Tom's password is equal to "pass". Someone malicious could simply enumerate through common passwords until the form redirects, at which point they know Tom's password. XCat is built to automate this, but it takes it a step further by being able to extract any portion of the document being queried through the injection flaw as efficiently as possible. XCat can also be used to read arbitrary XML and text files on the server - in the demo below we read an XML file, a secret text file and /etc/passwd.

Example command

You need to supply XCat with some information before it can exploit an injection flaw. It needs to know the HTTP method, the URI of the page, some data which triggers a True or False page, the vulnerable parameter and a match string. In the example below that is used in the demo the vulnerable parameter is "title", and if the query is successful (i.e evaluates to true) the resulting page will have "1 results found" inside the contents.

xcat --method=GET http://localhost:8080 "title=Foundation" title "1 results found" run retrieve

Using just this information XCat can retrieve the whole XML document being queried. For XCat to read local files and speed up retrieval it needs to know how to connect back to your local machine, which means you need a public IP address. In the video below I use the --public-ip flag to specify "localhost" as my address as I am running the example site on my local machine. You can set it to "autodetect" and XCat will automatically detect your public IP. Note: Maximize the demo (bottom right) if you can't see all the commands.

XCat comes with an example vulnerable website you can test it against, powered by Jython. It also tries to be clever about how it fetches data: It can use a variety of different methods to speed up retrieval and intelligently degrades to slower techniques if faster ones are not available. This means you just need to point XCat at the vulnerability and it will do the hard work of figuring out the best way to exploit it.

Installation

XCat requires Python 3.2 or above (preferably 3.4) and can be installed via Pip:

pip install xcat

The example application can be downloaded from the Github repository

Out of band attacks

Out-of-band (OOB) metaphorically refers to activity outside of some other kind of activity. In the examples above one request can get only a True or False response, so for each request we can only gather one bit of data. We can get around this by using an OOB attack to gather many bits of data per request by utilizing an additional communication channel.

The first OOB technique utilizes the XPath doc() function to speed up retrieval. This function can be used to load both local and remote (via HTTP) XML documents as part of a query:

doc('file://file.xml')/some/xpath/query

If XCat detects it is able to use this function it performs the following actions to speed up retrieval:

  1. Issue a query requesting the server to load an XML document from a URL, which contains part of the data we wish to extract concatenated onto the end of the URL (URL + urlencode(data))
  2. The server will request this URL and XCat can extract the information from the query string

It does this by starting its own HTTP server and listening for requests from the target server.

The second OOB technique allows the reading of local text files using XML External Entity Injection. It's quite similar to the first OOB attack but involves two stages: first XCat asks the server to load a malicious XML file which contains an XML SYSTEM entity pointing to the file we want to read, and secondly the resulting data is then sent via the first technique back to XCat. These are the steps taken:

  1. Instruct the server to load a crafted XML document served by XCat
  2. The server connects to XCat and requests the XML document
  3. XCat responds with a malicious XML document containing a SYSTEM entity
  4. The server parses this and includes the contents of the target file in the document
  5. This can be extracted and read by using the first technique

Check out the documentation for more info and examples of commands. A fair bit is missing at the moment, most noticeably support for cookies, but those should be coming at a later date.

A test RSS feed service

The coursework set for my Distributed Systems involves reading new items from RSS feeds (such as the BBC News feed or the UK traffic incident feed). To help me build the system I developed a simple service that serves up RSS feeds that are regularly automatically updated with nonsense items, and it might be useful for anyone else doing something related. It sure beats waiting for BBC news to publish a story or someone to crash on the M4.

The service can be found here: http://rss.tomforb.es/ and is really simple to use.

Basic feeds

A feed is accessed by a unique key. This key can be anything, e.g http://rss.tomforb.es/feed/rss/somefeed or http://rss.tomforb.es/feed/atom/blah123. If you can't think of a key (or don't want to hard code one while testing) then you can use http://rss.tomforb.es/feed/rss/cookie, requesting this will set a cookie which must be re-sent with each request to get the same feed.

Message rate

Messages will be added to the feed once every 30 seconds, but this is configurable with the ?time parameter. To create a feed with a new entry every 15 seconds request this URL: http://rss.tomforb.es/feed/rss/df2c0fbcd2814c288973a804c60df857?time=15

Simulate issues

Appending ?tryme to any feed URL will trigger a fault 25% of the time. This fault will be one of:

  • Invalid XML response
  • Internal server error (http 500)
  • Non XML response
  • Non HTTP response
  • Redirect to Google
  • Out of order or missing results
  • Results with future dates

You can use this to ensure any system you build is proofed against one of those faults.

Code

You can check out the source code here on GitHub

2 years of blogging

When I first came to University lots of people (like Rob Miles) were trying to get undergraduates to start blogging. On the 6th of March 2012 I registered this domain and started blogging, getting myself added to the awesome Hull Compsci blogs syndicate. That was two years ago and a lot has changed so I thought I would write a summary post.

Open source stuff

My first blog was a Wordpress one, but I felt that Wordpress was trying to be too much and I just wanted something a bit more lightweight. So, inspired by svbtle I created and released Simple which powers this blog. A least a few other people use it for their blogs as well which is kind of cool depite me not giving the project much attention recently.

I primarily use GitHub to share all my open source code. Over the last year I've made 379 contributions to both my own repositories and others that I use, and somehow gathered 48 followers. My most popular repository is Simple which has 481 stars and 96(!) forks, followed by my Django Debug toolbar panel with 191 stars.

I've put a few projects on the Python package index as well, and a couple seem to be used a fair bit: The Debug toolbar panel had 3690 downloads in the last month and my HtmlToWord library had 3812 (which I find weird because unlike the other package I can't locate a single project other than mine that actually uses it).

In the last two years this blog has had 67,184 visitors with a peak of over 1,000 concurrent viewers, which is pretty neat. Simple handled every request speedily with no errors.

The most popular posts are:

  1. Purchasing a £30,000 numberplate for the price of a bus ticket
  2. Breaking out of secured Python environments
  3. Just how slow are Django templates
  4. Opera is really nice
  5. Automatically inline Python function calls
  6. Using Python metaclasses to make awesome Django model field choices
  7. Adding tail call optimization to Python

According to Google Webmaster tools some of my popular posts have quite high google rankings for some queries:

Viewer information

I use Google Analytics for my website stats and it also tracks things like browser usage, operating system and screen resolution.

Security stuff

I found a 4 "grave" security issues in some software that got issued their own CVE numbers (which is awesome!): osvdb.org/creditees/11331-tom-forbes. Slightly before I started this blog I also presented a research paper at the BlackHat security conference which was the most scary hour of my life. I haven't really blogged about it but I intend to in the coming weeks.

Conclusion

I'm pretty happy with the last two years and I'm both excited for the future and a bit sad that my time at Hull is coming to a close.

Opera is really nice

I really like the Opera browser.

A couple of months ago I got a bit tired of using Google Chrome, it was just a bit sluggish sometimes and I fancied a change. So I switched to Firefox, which I enjoyed using for a month or so until it too became irksome - it used a hell of a lot of memory and was also often very sluggish (moreso than Chrome was). In desperation I decided to give Opera a spin and to my surprise I really liked it.

Opera had always been something I was aware existed but had never looked into. To be honest I didn't really consider it to be a real browser (how could it be with less than 4% market share?) and when one of my business partners told me he used it I felt a feeling of contempt: this guy hasn't seen the light that is Chrome and is stuck using some dated crummy browser. What a chump.

How wrong I was. Opera is fast, clean, responsive and is a pleasure to use. Since 2013 Opera has used the Blink layout engine which is the same one that powers Chrome and so it inherits some of Chromes features like a multi-process model, it's dev tools, sandboxing and even compatibility with its extensions. Talking of extensions I was pleasantly surprised at the number available - over 600 are listed on the add-on site with all of the popular ones (Adblock, LastPass, Reddit enhancement, Ghostery, etc).

It's not all roses though, I installed the Opera Next edition which is slightly more bleeding edge than the standard Opera release. The only issues I have had so far is when I have a large number of tabs that use Flash open. That tends to kill the browser, but it promptly restarts and re-opens all my tabs. Oh, and dragging a tab from one window to another is a little bit tricky, I think it has latched onto the new window but when I release it a new window opens (you just have to hold it there a bit longer than FireFox or Chrome). Neither of those are particularly critical and I can live with them.

If you are a bit tired of Chrome and want to try something different then I highly recommend Opera. Go give it a try.