Tag Archives: python

logo

Importing a comma-separated csv file into Numbers Tool

I created this to scratch an itch of mine when reading a csv file into Numbers that contained commas. It turned out that Numbers puts each line into a single cell. It ignores the commas. And there’s no way to specify that the delimiter needs to be a comma, not a semi-colon.

Here’s a small tool made using a small python script and Platypus that will launch as a ‘droplet’ and will allow you to drop a csv file on it that has COMMA (meaning this , ) separated values.

It will accept any csv file and convert from commas (,) to semi-colons (;) and save the result out again as “filename2.csv” – Numbers should be able to open it correctly then. Your original csv file is only read.

It works on my smallish csv files, but please note :

! WARNING ! THERE ARE NO GUARANTEES THAT THIS WILL WORK FOR YOU – YMMV ! WARNING !

Please note there is NO feedback if something goes wrong.

It works on my 2 macs (using OS X Mavericks), so I figured I would throw it out in the wild for those people who also might need it.

You can download the (non-signed) app here : Com2DotCom.app – you’ll need to open it using RIGHT-CLICKING on the App the first time.

Feedback, comments, bug reports, etc are very welcome. If possible, I’ll try to respond and help you out.

Fun Project : visualisation of all the flemish schools

I’ve had a bit of fun recently with visualising some open data from onderwijs.vlaanderen.be where I downloaded the address list of flemish schools for kindergarten and “lower” schools.

  • imported the csv list via “base” in an sqlite database
  • added longitude and latitude fields to the db
  • wrote a small script that used geopy to query google for each address and add the coordinates to the db
  • then created another small script using pygmaps to create a custom html file which includes the necessary coordinates for each point on the google map.

The result is that I never knew how many schools there were in Vlaanderen !

Schools in Flemish Belgium

You can read more in the original post (in Flemish) that I made on the dataconnect website, where I advertise for my web site design and consultancy. For the full map, click on the above image to go there.

Woes in Python 2.x : UnicodeEncodeError : ‘ascii’ codec can’t encode

Phew ! This is a reminder to myself…

I’ve wrapped my head once already around unicode, but it was so long ago that I had to do it all over again when the error suddenly came up again.

The error I had was the dreaded “UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa1′ in position 0: ordinal not in range(128)” error which does not really explain what you are doing wrong but leaves you scratching your head wondering where in heaven’s name you should start looking.

I read the following articles :

But what really solved the problem for me was this StackOverflow answer by Alex Martelli (Bless You Joel Spolsky for this idea and Bless all the people answering there!)

I was in a similar position code-wise : I had a unicode object that I wanted to convert tot utf-8, but I was using the “decode” function.

I actually should have been using the “encode to <codec>” function !

Duh ! Once you have a unicode object, you need to encode it into utf-8 or whatever encoding you need. I was using decode !!!

Oh. Well, perhaps my unicode fu is still not as good as it should be…

Proof of concept for a simple webserver running python code

Here is a small code example using CherryPy to run a very simple webserver that

  • generates a simple math question
  • compares the answer to the solution.

It’s meant as a proof of concept, so there is no security built in. It’s running on localhost on port 8888 (modifiable in the main part of the code).
It allows you to play around and test out your ideas.

Do not use this code on an outside network !

It’s simply an example showing how easy it is to set up a web server and how you can create pages for it using python and CherryPy. It’s been cobbled together in an evening from previous programming so there’s some cruft left in. I’ve also extensively commented the code.

Requirements:

  • python 2.7 ( 2.5 will work as well is my guess )
  • cherrypy 3.2.2 ( use easy_install or pip to download and install the latest version)
  • site.py ( the file containing the python code )

You start the server in a command prompt using : python site.py which will start the server. Leave the command prompt open.

You can then visit the webserver by opening a browser and going to http://localhost:8888 to see the index page and play around with it.

#
# MathPoc : Proof of concept of a simple math problem, bringing it to the browser
#
# Alex Boschmans
#
# Version 0.2, February 2011
#
# 0.2 Added some error checks and expanded math to not just adding but also 
# subtraction and multiplication and divisions. Extensively commented code.

header = """<HTML>
            <HEAD>
                <title>MATH Proof of Concept</title>
            </HEAD>
            <BODY>
        """
footer = "</BODY></HTML>"

indexhtml = """
        <H1>Math Proof of Concept</H1>
        <p>Please answer the following question</p>
                <p>How much is %d %s %d ? </p>
                
                <form action="/response" method="post">
                Answer: <input type="text" name="answer" />
                <input type="hidden" name="number1" value="%d">
                <input type="hidden" name="number2" value="%d">
                <input type="hidden" name="operation" value="%s">
                <input type="submit" value="Submit" />
                </form>
                
        """

def generatequestion():
    # This generates the question that we will pose using the random function
    # Generate a random question using 2 random numbers between 1 and 10
    number1 = random.randint(1,10) 
    number2 = random.randint(1,10)
    # Now we choose an operatioin
    ops = ["+", "-", "x", "/"]
    operation = random.choice(ops)
    # Let's check the division
    if operation == "/":
        # Prevent divisions with remainders using the modulo operator
        # Using module on the two numbers evaluates to 0 when no remainder is present
        # While the modulo remainder is not equal to 0, generate two new numbers
        while number1 % number2 <> 0:
                number1 = random.randint(1,10) 
                number2 = random.randint(1,10)
    # Assemble the html, inputting the numbers in the foreseen places in the html
    # In a more extensive project, you would keep this html in a template file and 
    # call it with a dictionary of items that need to be filled in the template
    question = indexhtml % (number1, operation, number2, number1, number2, operation)
    # Add common html like header and footer - these are defined just once and reused
    # for each page
    html = header + question + footer
    # Return the completed html to the calling function (in this case index)
    return html

# This is the class that the cherrypy server uses and where you create the views that the 
# webuser sees. After each definition there is a <function>.exposed=True that indicates if the 
# webuser can see this page or not. 
class MathPoc:
    def index(self):
        # This is the main index page that is shown to the user when he first visits the site.
        # We create the page by calling the function generatequestion (which is outside the class
        # MathPoc but accessible and we show it to the user by 'return'ing the page 
        page = generatequestion()
        return page
        # The webuser will now see the page and will have a chance to enter an answer.
        # In the html form I've specified that the submitted result will go to the url "response"
        # I've added all the values I want to receive either as hidden values (eg the 
        # original numbers, the operation) or as part of the form (eg the answer)
    index.exposed = True
    
    def response(self, answer, number1, number2, operation):
        # First check if we received an answer or if the user submitted without an answer
        if answer:
            # Calculate our own answer ourselves and generate a response to the user
            # We receive strings, so convert them to integers using int()
            number1 = int(number1)
            number2 = int(number2)
            answer = int(answer)
            # Answer is dependent on operation
            if operation == "+":
                solution = number1 + number2
            elif operation == "-":
                solution = number1 - number2
            elif operation == "x":
                solution = number1 * number2
            else:
                solution = number1 / number2
            # See if the answer is correct and display according the result
            # Using templates, you could put all this in one template and 
            # call the template with options so it knows what to show
            if solution <> answer:
                html = """
                <H1>Sorry.</H1>
                <p>The question was : %s %s %s = ?</p>
                <p>Your answer %s is wrong. The correct answer is %d.</p>
                <p><a href = "/">Try Again.</a></p>
                """ % (number1, operation, number2, answer, solution)
            else:
                html = """
                <H1>Correct !</H1>
                <p>The question was : %s %s %s = %s</p>
                <p>Your answer is correct !</p>
                <p><a href = "/">Try Again.</a></p>
                """ % (number1, operation, number2, answer)
        else:
            # We did not receive an answer
            html = """
            <h1>Sorry ?</h1>
            <p>You need to fill in an answer !</p>
            <p><a href = "/">Try Again.</a></p>
            """
        # Return the page to the user, adding the common html
        return header + html + footer
    response.exposed = True

if __name__ == '__main__':
    import random
    import cherrypy
    import os, sys
    # Set the current directory - this is probably not needed for this example, cruft.
    try:
        current_dir = os.path.dirname(os.path.abspath(__file__))
    except:
        # probably running inside py2exe which doesn't set __file__
        current_dir = os.path.dirname(unicode(sys.executable, sys.getfilesystemencoding( )))

    # Set up site-wide config first so we get a log if errors occur.
    # Adding the setting 'environment': 'production' to the below turns off auto-reload.
    # Otherwise CherryPy monitors the code and any change to code reloads the server - handy for development !
    cherrypy.config.update({'server.socket_port':8888,
                            'server.socket_host':'127.0.0.1',
                            'log.error_file': 'site.log',
                            'log.screen': True})
    # CherryPy will complain of an empty config but will continue
    conf = {}
    cherrypy.tree.mount(MathPoc())
    #cherrypy.config.update({'server.socket_port':8888})
    cherrypy.quickstart(MathPoc(),'/', config=conf)