Our first Homebrew I

Finding the bottles was easy, cleaning off labels not so much; stainless steel wool and hot soapy water worked best. Sadly our first crop of hops was less than abundant, so I’ve resorted to all bought ingredients for now…

CPMG simulation/modeling

A script for simulating the CPMG sequence using the density matrix, with Gauss/Gauss envelope/spikelet broadening, for a forthcoming paper.

---

function [tot, gbb,t]=d_cpmg(N,tau,n,delta,r,rr)

%cpmg experiment evolution wjb 02/09
%simple/ideal pulse sequence:

%90y-n*[-180x-]

%N time steps
%tau
%n 180 loops
%delta chem shift
%r envelope gauss br^2
%rr spikelet gauss br^2

%matrix for I_x & I_y

a=[0 1/2; 1/2 0]; b=[0 -i/2; i/2 0];

%time step & initial rho

t=tau/(N-1); rho=a;

for k=1:n
sig(1)=trace(rho*a); sigi(1)=trace(rho*b);

for j=2:N/2

%iterate; free precession for tau/2

rho = [exp(-i*t*delta) 0; 0 exp(i*t*delta)]*rho*[exp(i*t*delta) 0; 0 exp(-i*t*delta)];

sig(j)=trace(rho*a); sigi(j)=trace(rho*b);

end

%apply 180x

rho = [0 exp(i*pi/2); exp(i*pi/2) 0]*rho*[0 exp(-i*pi/2); exp(-i*pi/2) 0];

%iterate; free precession for tau/2

sig(N/2+1)=trace(rho*a); sigi(N/2+1)=trace(rho*b);

for j=2:N/2

rho = [exp(-i*t*delta) 0; 0 exp(i*t*delta)]*rho*[exp(i*t*delta) 0; 0 exp(-i*t*delta)];

sig(j+N/2)=trace(rho*a); sigi(j+N/2)=trace(rho*b);
end

if k>1

tot=[tot (sig+i*sigi)];

else

tot=(sig+i*sigi);
end

end

tt=-tau/2:t:t*(N-1); gb=exp(-rr.*tt.^2); gb = [gb(N/2+1:N) gb(1:N/2)]; gbb=gb;

for i=1:n-1

gbb=[gbb gb];

end

t=0:t:t*(n*N-1); gbb=gbb.*exp(-r.*t.^2);

---

Glorious Gawk part II

Here’s a snapshot from a shell script to extract various important segments from a *ps file, after conversion from *pdf. It uses various gawk/awk tricks including using patterns for brackets, checking lengths of records to discriminate lines/polylines. Your mileage may vary a little, but if you check the *ps file preamble, you should be able to translate this to your specific tasks

---

#!/bin/sh
# line/text extract from pdf wjb 12/08, 02/08

NOFILE=64
[ -z $1 ] && echo “foo.sh <filename>” && exit $NOFILE

myfile=${1}

#convert to ps

echo “converting pdf -> ps…”

pdf2ps $myfile gHjLz.ps

echo “…done”

#take out line drawing sections w/ line numbers

echo “extracting lines & text…”

awk ‘$4==”scale”,$1==”Q” {print NR > “gHjLq.txt”}’ gHjLz.ps
awk ‘$4==”scale”,$1==”Q” {print $0 > “gHjLp.txt”}’ gHjLz.ps
awk ‘$1==”q” {print NR > “test.txt”}’ gHjLz.ps

# _p == polylines,  _l == lines

awk ‘BEGIN { RS = “q” } ; {if (NF > 12) print NR,$0 > “gHjLp_p.txt”; else print NR,$0 > “gHjLp_l.txt”}’  gHjLp.txt

awk ‘$4==”scale” {print $1 > “gHjLp_foo.txt”}’ gHjLp_p.txt

awk ‘$4==”scale”,$1==”Q” {print NR,$0 > “gHjLp_mol.txt”}’ gHjLz.ps

#take out text w/ line numbers

awk ‘$4 == “,” {print NR,$0 > “gHjLp_text.txt”}; $1 == “$C” {print NR,x > “gHjLp_text.txt”}; {x=$0}; $1 == “$C”, $1 == “,” {print NR,$0> “gHjLp_text.txt”}’ gHjLz.ps

Superb Debut by Wolf Parade

albeit in 2005; tracks 9 and 11 are brilliant

Building a distributed PSE

Over the last couple of years, I’ve been invested in building a distributed problem solving environment (PSE). I offer a simplified application here in the hopes that the general ideas can be useful to someone.

What has also initiated a desire to see this released is my wife’s iPhone with its addictive applications. On the downside, there is a serious lack of real estate and unless I convert to Mac I can’t get my hands on the SDK. However, coupling the iPhone’s portability with a personal server running traditional applications makes for a cheap and powerful PSE. Computational tasks and data storage can take place server side with the iPhone serving as a very pleasing web portal.

There are plenty of applications of these ideas to the sciences, for someone with a modicum of talent and vision. For the purposes of this example I’m going to use free financial data and the computational task will be options pricing under the Black Scholes model for European contracts. I’ve taken a data source at random, and I have no idea if there is a European style options contract available. Also realize that there is potentially unlimited risk associated with options, and it is widely held that the traditional theory is naïve and at worst downright wrong, particularly when using historical data to calculate volatility. So use this example under advisement…and as always, please be careful with curl, don’t get yourself banned

1. Set up MySQL + Apache + PHP on a server
2. Create Database + table on server:

//invoke mysql (as root)
>mysql –u root –p
//create database
>create database tech_stock;
//extend permission to user bill with password *****
>GRANT ALL ON tech_stock.* TO bill@localhost IDENTIFIED BY “*****“;
//quit;
>quit;
//as user, invoke mysql as before; select database
>mysql tech_stock –u bill –p
//create a table
>create table tech_A( n_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, date VARCHAR(8), open FLOAT,
high FLOAT, low FLOAT, close FLOAT, volume FLOAT);
//check it exists
>DESCRIBE tech_A;
//logout
>quit;

3. Write a shell script to grab data:

#!/bin/sh
# data update/download wjb 02/08
Month=$(date +%b)
Day=$(date +%d)
Year=$(date +%G)
str1=”http://finance.google.com/finance/historical?cid=659815&startdate=Feb+10%2C+2008&enddate=”
str2=”%2C+”$Year”&output=csv”
curl $str1$Month”+”$Day$str2 > tech_A.csv
exit #cheerio

4. Make a html portal:

<html><body style=”font-family:verdana”>
<title>PSE Demo</title>
<form action=”main.php” method=”post” enctype=”multipart/form-data”>
<font size=”5″ style=”color:blue”>Data Processing Portal<br>
<table style=”background-color:blue”>
<tr><td><font color=”white”>Password:</font> </td>
<td><input type=”password” name=”pswd” /> </td></tr>
<tr><td><font color=”white”>Email: </font></td>
<td><input type=”text” name=”email” /> </td></tr>
<tr><td><input type=”submit” name=”cmdupload” value=”Submit” /></tr></td>
</table>
</form>
<br>
<font size=”1″>
Collaboratory for SDE data modeling <br>
</font>
</body></html>

more to come…

Quick tour through Hilbert space

Before part III of forecasting with RSS+SVM+wavelets I thought it would help to give some useful concepts from Hilbert space. Attached is a very rough look, and also an application using orthogonal functions to model a stationary signal (Fourier series). This will contrast nicely with wavelets, which are most useful for non-stationary signals eg., stock indices.
hilbert space overview
Fourier series example

latest Walkmen album

sure, it’s been out a while, but I finally bought and have been enjoying again the melodic genius of The Walkmen in the form of You & Me

Mining Spectra + Molecules

I’ve been working with colleagues and collaborators from the UK to mine NMR spectra and corresponding molecular structures from documents. The object is to create an XML/CML database to give researchers unprecedented access to information, useful in (for instance) drug discovery. At this stage, I have focused on writing algorithms for the extraction of molecules to *svg, and NMR data to *txt. The latter is then refined and processed, first to determine peak positions. The data is then optimally fit using mixture models, and peak lists created automatically using standard and novel algorithms.