Category: programming

basic python gui

It feels like years since I posted something technical; I have a busy day job and at least one side gig to thank for that. Thought I’d share the following python code, handy for wrapping a terse command line app (myApp) in a slightly more user friendly/cross platform gui. Don’t forget to re-introduce the persnickety python formatting 😉


from tkinter import *
from tkinter import ttk
from tkinter import filedialog
import subprocess
import time
import os
root = Tk()

# initialize parameters
programArgs = (' --a ', ' --b ', ' --c ')
argumentNames = ('Input File', 'Output Log', 'Output Directory', 'Output Prefix')
initMessage = 'Required'
argumentMap = { }
processID = ''
baseCommand='myApp.exe'
pathSep='\\'

# adapt syntax to OS
if os.name == 'posix':
baseCommand='myApp.x'
pathSep='/'

# test availability of binary
testCommand=''
try:
testCommand = subprocess.check_output(baseCommand, stdout=subprocess.PIPE)
if "allowed options" not in testCommand.communicate()[0]:
print('please install the myApp executable and make sure your PATH is correct')
time.sleep(3)
root.quit
except:
pass

# select a file or directory
def openFileDir(entry,field):

f=StringVar()
if 'Dir' not in field:
if 'Input' not in field:
f = filedialog.asksaveasfilename()
else:
f = filedialog.askopenfilename()
else:
f = filedialog.askdirectory()

if len(f) > 1:
argumentMap[field] = f
entry.delete(0,END)
entry.insert(0,f)
else:
argumentMap[field] = ''
entry.delete(0,END)
entry.insert(0,initMessage)

# get entry for prefix
def updateArgument(entry,field):

f=entry.get()

if "Required" not in f:
argumentMap[field]=f
else:
argumentMap[field]=''

# inform the user, uninitialized params
def printArgumentError(f):
msg="Please set a value for the parameter: "
msg+=f
print("%s" % (msg))

# create command
def createCommand():

v=StringVar()
comm=baseCommand + programArgs[0];
v=argumentMap[argumentNames[0]]
if len(v) <= 1:
printArgumentError(argumentNames[0])
return ''

comm+=v
comm+=programArgs[1];
v=argumentMap[argumentNames[1]]
if len(v) <= 1:
printArgumentError(argumentNames[1])
return ''

comm+=v
comm+=programArgs[2];
v=argumentMap[argumentNames[2]]
if len(v) <= 1:
printArgumentError(argumentNames[2])
return ''

comm+=v
comm+=pathSep
updateArgument(e4,argumentNames[3])
v=argumentMap[argumentNames[3]]
if len(v) <= 1:
printArgumentError(argumentNames[3])
return ''

comm+=v
print('%s' % comm)
time.sleep(10)
return comm

# run command
def runCommand():

comm=StringVar()
comm=createCommand()

if len(comm) <= 1:
msg="Can't execute myApp; missing required parameters"
print('%s' % msg)
return

subprocess.Popen(comm, stdout=subprocess.PIPE)

# create and grid the form
c = ttk.Frame(root)
c.grid(sticky="nsew")
c.master.title("myApp")

# create the widgets, first grid
# label, entry and browser for input
f1 = argumentNames[0];
l1 = ttk.Label(c, width=20, text=f1)
e1 = ttk.Entry(c,width=30)
e1.insert(0, initMessage)
b1 = ttk.Button(c, text='Browse',
command=lambda: openFileDir(e1,f1))
argumentMap[f1] = ' '

# label, entry and browser for output log
f2 = argumentNames[1]
l2 = ttk.Label(c, width=20, text=f2)
e2 = ttk.Entry(c,width=30)
e2.insert(0, initMessage)
b2 = ttk.Button(c, text='Browse',
command=lambda: openFileDir(e2,f2))
argumentMap[f2] = ' '

# label, entry and browser for output file
f3 = argumentNames[2]
l3 = ttk.Label(c, width=20, text=f3)
e3 = ttk.Entry(c,width=30)
e3.insert(0, initMessage)
b3 = ttk.Button(c, text='Browse',
command=lambda: openFileDir(e3,f3))
argumentMap[f3] = ' '

# label, entry for prefix
f4 = argumentNames[3]
l4 = ttk.Label(c, width=20, text=f4)
e4 = ttk.Entry(c,width=30)
e4.insert(0, initMessage)
argumentMap[f4] = ' '

# grid all the widgets
# first row, input
l1.grid(row=0, column=0, padx=5, pady=10)
e1.grid(row=0, column=1, pady=10)
b1.grid(row=0, column=2, padx=5)

# second row, output log
l2.grid(row=1, column=0, padx=5, pady=2)
e2.grid(row=1, column=1, pady=10)
b2.grid(row=1, column=2, padx=5)

# third row, output ldf
l3.grid(row=2, column=0, padx=5, pady=10)
e3.grid(row=2, column=1)
b3.grid(row=2, column=2, padx=5, pady=2)

# fourth row, output prefix
l4.grid(row=3, column=0, padx=5, pady=10)
e4.grid(row=3, column=1)

c.grid_columnconfigure(0, weight=1)
c.grid_columnconfigure(1, weight=1)
c.grid_columnconfigure(2, weight=1)
c.grid_rowconfigure(0, weight=1)
c.grid_rowconfigure(1, weight=1)
c.grid_rowconfigure(2, weight=1)
c.grid_rowconfigure(3, weight=1)

# second grid, action buttons
d = ttk.Frame(root)
d.grid(sticky="nsew")
b4 = Button(d, text='Quit', command=root.quit)
b5 = Button(d, text='Start', command=lambda: runCommand(),bg="green")
b6 = Button(d, text='Stop', command=root.quit,bg="red")
b4.grid(row=0, column=0, padx=5, pady=2)
b5.grid(row=0, column=1, padx=5, pady=2)
b6.grid(row=0, column=2, padx=5, pady=2)

# fix size
root.update()
root.minsize(root.winfo_width(), root.winfo_height())
root.maxsize(root.winfo_width(), root.winfo_height())

# run
root.mainloop()

 

Advertisements

new GPU book

Numerical Computations with GPUs comes out later this year; Pierre-Yves and I were able to contribute a chapter on LU &QR decomposition (the latter using Givens rotations) for batches of dense matrices. We saw some impressive performance improvements for specific problem sizes. QR will benefit particularly from CUDA 6 and the availability of the fast/safe reciprocal hypotenuse function rhypot(x,y), more details here .

HPC Essentials 0

The notes from my last talk at PSU for the forseeable future, delivered to the Math department during colloquia a few weeks back. A kind of prequel to the HPC Essentials series, I take a simple Kriging process through the steps of making it an example of high performance computation. Lots of information, perhaps too much 🙂

cluster profiler

We’ve been working on a method to effectively monitor and in some senses profile all relevant processes running on one or more systems. An alpha has been released on github, Pierre-yves is working on a powerful flume+solr component for search, readme follows, more to come.


Overview

This code comprises the clpr_d daemon for the Cluster Profiler project, an Orwellian attempt to develop time series and statistics for all running processes on a single system or many systems. Process data is gathered or clustered according to process birthdate (rounded to the minute) and uid. The daemon uses several threads to work on a boost::multi_index data structure, containing the acquired process data. The main thread reads from a named pipe specified in key_defines.h, data produced by running and re-directing output from pidstat in (eg.,) an external shell loop. Appropriately formatted pidstat output may be produced from this forked code, the utility originally produced by Sebastien Godard : https://github.com/wjb19/sysstat. An example usage is the following, removing process data from root, and redirecting to a fifo in the bin directory of this distribution :

pidstat -d -u -h -l -r -w -U -v | grep -v root > bin/clpr_input

An archiving thread works asynchronously to write port 80 using tcp/ipv4 whenever queried, according to the format specified in the ostream operator for clpr_proc_db, the wrapper around boost::multi_index. Acquired process data from the fifo is ‘blobbed’ together by the reader and statistics developed on the fly. A logging class will write a log file periodically as well, with filename specified in key_defines.h. Top utilization statistics are reported in the log file, using the multiple search/sort indices of boost::multi_index. Finally, a manager thread periodically monitors the size of the database, trimming/deleting entries according to timestamps and a maximum size.

Keep in mind this daemon can be a security risk and compute as well as i/o intensive. It has been designed with flume + solr in mind; for the overall project, flume is used to query various instances of clpr_d, and solr used for indexing and search on records – WJB 03/14
Installation

Install fork of pidstat specified, ‘make’ this distribution, specifying compile and linker paths for boost as needed.

avpipe alpha

While I nut out the details of working with the new open source codec for h.264 from cisco, I’ve gone ahead and released the code for the aforementioned avi processing application, tentatively dubbed avpipe, on github . Performance is fairly good, although memory management needs attention at some point 🙂

hpc app for avi

There are many good open source tools for processing video data, but very little in a HPC context; and yet processing image data/video streams is becoming increasingly common, at least from the perspective of our unit. Hence I’ve invested a little time in an application that works directly on the stream from stdin, using FFmpeg, OpenCV and boost::threads. With the power of UNIX pipes, different processing algorithms are readily concatenated, all without the need for copious i/o. Hexdump -C has been a lifesaver in terms of discovering the quirks of the format.


00000000  52 49 46 46 48 e0 6f 3e  41 56 49 20 4c 49 53 54  |RIFFH.o>AVI LIST|
00000010  88 22 00 00 68 64 72 6c  61 76 69 68 38 00 00 00  |."..hdrlavih8...|
00000020  21 00 00 00 5c 05 51 00  00 00 00 00 10 09 00 00  |!...\.Q.........|
00000030  d5 0a 4d 00 00 00 00 00  02 00 00 00 00 00 10 00  |..M.............|
00000040  80 07 00 00 38 04 00 00  00 00 00 00 00 00 00 00  |....8...........|
00000050  00 00 00 00 00 00 00 00  4c 49 53 54 ae 10 00 00  |........LIST....|
00000060  73 74 72 6c 73 74 72 68  38 00 00 00 76 69 64 73  |strlstrh8...vids|
00000070  61 76 63 31 00 00 00 00  00 00 00 00 00 00 00 00  |avc1............|
00000080  01 00 00 00 30 75 00 00  00 00 00 00 d5 0a 4d 00  |....0u........M.|
00000090  00 00 10 00 ff ff ff ff  00 00 00 00 00 00 00 00  |................|
000000a0  80 07 38 04 73 74 72 66  46 00 00 00 45 00 00 00  |..8.strfF...E...|
000000b0  80 07 00 00 38 04 00 00  01 00 18 00 61 76 63 31  |....8.......avc1|
000000c0  00 ec 5e 00 00 00 00 00  00 00 00 00 00 00 00 00  |..^.............|
000000d0  00 00 00 00 01 42 e0 32  ff e1 00 0e 67 42 e0 32  |.....B.2....gB.2|
000000e0  da 01 e0 08 9a 6e 02 02  0c 04 01 00 04 68 ce 3c  |.....n.......h.<|
000000f0  80 00 4a 55 4e 4b 14 10  00 00 04 00 00 00 00 00  |..JUNK..........|
00000100  00 00 30 30 64 63 00 00  00 00 00 00 00 00 00 00  |..00dc..........|
00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|