ALL YOUR BASE ARE BELONG TO US
Physics. Data Science. General Geekery. Probably Coffee.
This is a little intro to using an LSTM model for time series data. It's not in any way a thorough introduction to how LSTMs work, which is pretty complex and far too much info for a short blog like this one... I'm going to demonstrate it using some financial (stock market) data, where I'll …
Continue reading "FinTechAI: Predicting Adjusted Close Values using LSTMs"
In their classic 2012 paper: Improving neural networks by preventing co-adaptation of feature detectors, Hinton, Srivastava, Krizhevsky, Sutskever and Salakhutdinov showed that using dropout in a feed-forward neural network improved performance significantly. In particular they produced this result for the MNIST dataset: However, if you try to create this result in PyTorch using modern standards …
A while ago I wrote about how to extract text from PDF documents in Python using the PDFMiner library. However, in a recent project I had some trouble using PDFMiner to extract text, possibly because the documents I was working with were scanned PDFs. In this case the answer is to use OCR-based text extraction, …
In my previous post I talked about how to use random forest classification to separate true pulsar candidates from RFI. That classification used numerical features extracted from the processed data. Ultimately it would be interesting to just be able to use the data itself, rather than extracted features. To help me do that I've created …
Pulsar classification is a great example of where machine learning can be used beneficially in astrophysics. It's not the most straightforward classification problem, but here I'm going to outline the basics using the scikit-learn random forest classifier. This post was inspired by Rob Lyon's pulsar classification tutorials in the IAU OAD Data Science Toolkit. This post …
Recently I've been playing around with Mapbox. I previously used it to make the visualisation in my post on the UK gender pay gap and while I was doing that I started wondering about using choropleths. The layers you need to make choropleths of the USA are available within Mapbox, but I couldn't find any …
I have been wondering how the gender pay gap in the university sector compares with the average across other sectors, so I thought I'd look at the data. Last update: 23 March 2018. The data I'm using come from the current UK returns on gender pay differences. Every company in the UK with more then 250 …
I love Taipei. I also love Open Data. So I was very happy to read that the National Palace Museum in Taipei had an open data project. According to the article, the museum has put images and meta-data for 70,000 items online. So what do you get if you download the information on a particular …
Continue reading "Night at the Museum: Translation in Python"
Tired of reading all those documents everyone keeps sending you? Why not get your Jupyter Notebook to do it for you and condense the information? I'm joking of course... but if say you did want to read pdf documents directly in Python, how would you do it? Recently I had a go at doing just …
Recently I got hold of a PYNQ-Z1 board and accessories kit from Digilent via the Xilinx University Program. This nifty piece of kit supposedly lets you program an FPGA (the Xilinx ZYNQ) using a Jupyter notebook. What's in the Box? Here's a picture of what's inside: Getting Started I'm following the PYNQ Guide to Getting …
ALL YOUR BASE ARE BELONG TO US 