Entries : Category [ Django ]
The Django web development framework
[OpenBSD]  [BSD]  [FreeBSD]  [Linux]  [Security]  [Python]  [Zope]  [Daily]  [e-shell]  [Hacks]  [PostgreSQL]  [OSX]  [Nintendo DS]  [enlightenment]  [Apache]  [Nintendo Wii]  [Django]  [Music]  [Plone]  [Varnish]  [Lugo]  [Sendmail]  [europython]  [Cherokee]  [self]  [Nature]  [Hiking]  [uwsgi]  [nginx]  [cycling]  [Networking]  [DNS] 

25 noviembre
2008

django i18n support: fuzzy!

well, it is not a matter of django, but of gettext anyway...

It took me some time until I discovered this, so I think it would be nice to post about it.

In a recent project, I needed to add i18n support to a django app. This was the first project that needed such support since the first time i used django some time ago, so I took a look at the official docs, I read some posts in some blogs over the internet and I followed three easy steps:

1- Inside my app dir, I used the manage.py tool to generate the needed .po files for my language (.es):

../manage.py makemessages -l es

2- Then I opened up emacs and I used the po mode to edit the .po file (locale/es/LC_MESSAGES/django.po inside my app dir). It is pretty easy to edit a .po file, you could use any text editor to add the translated strings in place, but the po mode has some interesting shortcuts to get it done quickly.

3- Finally I generated the .mo file that contains the pre-compiled translation information, again, using manage.py inside my app dir:

../manage.py compilemessages

Then I restarted my django server (I would have to restart apache if we have the app in a production environment) and I tested the app on a web browser.

Nice! It seemed to work, everything was in .es now... but... wait... there were still some untranslated texts.

I went back to emacs and I revised the whole .po file, everythin seemed to be ok, I regenerated the .mo file and restarted the server again... and those strings were still untranslated.

Then something catched my eye in the .po file, the word fuzzy:

#: models.py:75
#, fuzzy
msgid "Public states"
msgstr "Publicados"

Seems like the fuzzy mark prevents the strings marked as fuzzy to be translated, as that means it is marked for translator revision (more info: http://www.gnu.org/software/automake/manual/gettext/Fuzzy-Entries.html#Fuzzy-Entries ).

To solve this problem, I just removed the fuzzy mark, I regenerated the .mo file and I restarted the server once more. And then everything was correctly translated.

Posted by wu at 23:06 | Comments (0) | Trackbacks (0)
27 noviembre
2008

django verbose_name limitation

Learning new things every day

This is one of those things you never know about until it happens to you.

Some days ago, while working on a django project, I made some changes to an app models. Since I was at the beginning stages of the development process, I just droped the database, created it again and I used syncdb to recreate the whole thing (yes, I used syncdb because I didn't use nor django-evolution neither south or any other migrations tool).

It seemed to be working properly:

snowball:Zero14 wu$ python manage.py syncdb
Creating table auth_permission
Creating table auth_group
Creating table auth_user
Creating table auth_message

But after created the default user to access the admin interface, I got this error:

psycopg2.ProgrammingError: value too long for type character varying(50)

(full trace here).

It seems like something (probably in models.py) was wrong. I revised my models file, nothing about a field limited to 50 characters... strange.

As usual, I asked in #django (on freenode) and, as usual, Magus- pointed me in the right direction, asking me if I had any model with a name longer than 50 characters. Of course I do not have any model with such a long name, but searching a little bit deeper, I found that one of the models had a verbose_name had a 80-character long string associated.

Replacing that with a more appropiate string (below 50 characters) fixed the problem.

Posted by wu at 09:19 | Comments (0) | Trackbacks (0)
22 enero
2009

django forms and where to add dynamic choices

is it forms.py, is it views.py, where?

In django, you can use the so-called ModelForms to generate a form directly from a model definition. This is pretty useful, as you can have a models.py file like, for example:

class Project(models.Model):
    name = models.CharField('Name of the project', max_length=100)
    code = models.CharField('Code number of the project', max_length=10)
    creator = models.ForeignKey(User, related_name='created_projects')
    maintainer = models.ManyToManyField(User, related_name='maintained_projects')

In this case we are creating a Project model, that is, a model to define information we will handle for those projects (this is an example, of course it would have more fields). The first two fields are CharFields (text fields where we will save the name and a numeric code for the project). The next two fields are ForeignKey and ManyToManyField fields (links to another model, this time User, which is django.contrib.auth.models.User, that is, the default django user model).

I've added two different relations to the User model, one will be the creator of the project, the other one the maintainer (or maintainers) of the project.

Now, back to ModelForms, I can create an HTML form just adding something like that to the app forms.py:

from myproject.myapp.models import Project

class ProjectForm(forms.ModelForm):
    class Meta:
        model = Proyecto

Once you've defined the form, you can get an html form just doing:

from myproject.myapp.forms import ProjectForm

myform = ProjectForm()

and you can use the form as described in the forms documentation (I'll not step there in detail).

But there is one problem with that approach. Imagine I've created two different groups (using django.contrib.auth.models.Group), let's call them project creators and project maintainers. Only users in the project creators group should be available in the creator field and users in the project maintainers group should be available to select for the the maintainer field.

Well, that's easy, we only have to modify a bit our forms.py:

from myproject.myapp.models import Project
from django.contrib.auth.models import Group

class ProjectForm(forms.ModelForm):

    creator_choices = [(c.id, c.username) for c in Group.objects.get(name__icontains='creator').user_set.all()]
    maintainer_choices = [(m.id, m.username) for m in Group.objects.get(name__icontains='maintainer').user_set.all()]

    creator = forms.ChoiceField(required=True, label='Project creator', choices=creator_choices)
    maintainer = forms.MultipleChoiceField(required=True, label='Project maintainer(s)', choices=maintainer_choices)

    class Meta:
        model = Proyecto

Ok, what we've got here is that I've overwrote or replaced the default creator and maintainer fields automatically generated by django, setting two new fields. This allow me to set certain default options for the fields, like the list of choices available (which I've generated searching for users in the needed groups).

Now a user will be able to select only users in the project creators group in the creator field and users in the project maintainers group in the maintainer field.

BUT, there is a problem with that. Doing it that way, the list of choices for each field will be generated on startup time (when you launch the django development server or apache or lighttpd or nginx or whatever you are using). That means that if you add a new user to the maintainers group, it will not appear in the maintainer field until you restart your server!

To avoid that, we will need to add the current available choices before using the form, overwriting the default list of choices:

from myproject.myapp.forms import ProjectForm
from django.contrib.auth.models import Group

creator_choices = [(c.id, c.username) for c in Group.objects.get(name__icontains='creator').user_set.all()]
maintainer_choices = [(m.id, m.username) for m in Group.objects.get(name__icontains='maintainer').user_set.all()]

creator = forms.ChoiceField(required=True, label='Project creator', choices=creator_choices)
maintainer = forms.MultipleChoiceField(required=True, label='Project maintainer(s)', choices=maintainer_choices)

myform = ProjectForm()

myform.fields['creator'].choices = creator_choices
myform.fields['maintainer'].choices = maintainer_choices

And that's it! This way the auto-generated html form will show up-to-date information in the select combos.

Posted by wu at 13:24 | Comments (0) | Trackbacks (0)
18 junio
2009

Create your own pony with magical powers

This pony-thing has finally made it!

Someone sent yesterday a link to mylittledjango.com to the europython mailing and I couldn't resist myself to take a look... This is the result, after some tests:

shellie! the e-shell.org pony with magical powers!

I have to say that I like the idea of the django pony since I heard about it for the first time, bud I didn't like the default colors chosen for the pony (pink?? what about the usual django green?). I really like the drawing, so having a web to generate a copy of the pony with your own custom/favourite colors on it... awesome!!

Now, go and get you one!

UPDATE:

I've see a lot of then in the pony gallery, but this is the funniest one of them all!

From Grayskull import Powers!

from Grayskull import Powers ;D

Posted by wu at 23:58 | Comments (0) | Trackbacks (0)
26 octubre
2015

Django, SQLite, GLOB, CAST and sorting

... or how to sort (properly) by a char field containing number values

Some days ago I found an interesting dilemma while working on a django based application. The application has a model similar to this one:

class Customer(models.Model):
    code = models.CharField(_('Customer code'), max_length=10)
    joined = models.DateField(_('Joined date'), auto_now_add=True)
    first_name = models.CharField(_('First name'), max_length=255)
    last_name = models.CharField(_('First name'), max_length=255)
    email = models.EmailField(_('Email address'))

    ...

That code attribute is a CharField where alphanumeric customer codes are stored. Those codes are mostly numbers (1,2,3...N) but then there are also codes like 2a, 3-groupB, etc.

At one point, it became useful to sort such users based on that code. If that would be an IntegerField, it would be as easy as using order_by:

from .models import Customer
customers = Customer.objects.all().order_by('code')

Or, if you want the results sorted in descending order:

customers = Customer.objects.all().order_by('-code')

But then, this was not an IntegerField, but a CharField, which means that using order_by would return something like:

0
1
10
11
12
13
...
2
20
21
...
3
...

And so on. Not exactly what was needed.

The following is the write-up covering the little adventure that was the work to fix it.

My first thought was - hey, this is a string, it will be properly sorted if we just fill the numbers properly -. Using python's zfill it would be really easy to calculate the larger value, and then pad zeros to the smaller values and sort them.

Sounded good, but you can't tell django's ORM to sort by a model method or property. So that approach would mean writing some code to sort the results in python code, after retrieving them from the database. Something you could do, but that is not exactly a good choice (and then you would have to parse codes a bit, as they are not numeric only).

Doing it at database level sounded like the way to go. The database should be able to handle sorting efficiently, no need to write some more code at application level.

Thinking a bit more about it I did recall of the CAST function from PostgreSQL, that can be used to change dinamically the type of a field, for example when making a query. This means that instead of a query like this, that will sort the results incorrectly:

select * from customer order by code;

We could do something like:

select * from customer order by CAST(code as int);

Which would first transform codes into integers, then order the results by that. Nice! - I thought - That should do it. But it did not.

Remember that code has values that cannot be directly casted to an integer (3-groupB for example). That means that if you run that query, you will get an error like this one:

ERROR:  invalid input syntax for integer: "3-groupB"

Here is where the substring function from PostgreSQL becomes handy. Using such function, you can extract parts of a string/char value, matching patterns, even using some regexps, as it is needed in this case.

Something like that should do it:

select * from customer order by CAST(
    substring(code from '^[0-9]+')
    as int);

There, substring extracts characters from code that match numbers from 0 to 9, CAST changes that resulting string into an integer and the query orders the results by that.

"Got it" - I thought. "Now, let's tell django's ORM to use that".

That was the easy part. Django makes it really easy with the extra method of django QuerySets. Something like this should do it:

customers = Customer.objects.extra(select={
    'sorted_code': "CAST(substring(code from '^[0-9]+') as int)"}
).order_by('sorted_code')

"Great, that was easy, let's give it a try" - and then I opened a django shell and gave it a try, but all I got was a huge traceback ending with:

OperationalError: near "from": syntax error

"W T F" - I did exclaim. I did check the syntax like a thousand times, and it would have worked flawlessly on any PostgreSQL server. I was sure of that. And that was precisely the problem, for this project we have been using SQLite instead. FACEPALM.

Anyway, SQLite is good enough and should be able to do this too, maybe just using a slightly different syntax, right?

In SQLite we can use GLOB (instead of pgsql's substring) and then its own CAST implementation, both inside a CASE expression, to mimic what we did with pgsql. This SQL code should work:

select * from customer order by
case
  when code GLOB '[0-9]*' then cast(code as int)
  else code
end

This is a bit more "complete" than the previous code we used in PostgreSQL, as it has a fallback in case GLOB do not find any match to be casted, returning the value of code as it comes from the db [1].

No, let's translate this into django extra code:

cast = """
CASE
  WHEN code GLOB '[0-9]*' THEN CAST(code AS INT)
  ELSE code
END
"""
customers = Customer.objects.extra(
    select={'casted_code': cast}
).order_by('-casted_code')

Et Voilà. That code could be used anywhere in your app. In my case, I had to put it inside the get_queryset of a ModelAdmin subclass. This way the django admin shows customers sorted correctly by default:

class CustomerAdmin(admin.ModelAdmin):

    # here goes the usual inlines, list_display, list_filter, etc

    def get_queryset(self, request):
        qs = super(CustomerAdmin, self).get_queryset(request)
        cast = """
        CASE
          WHEN code GLOB '[0-9]*' THEN CAST(code AS INT)
          ELSE code
        END
        """
        qs = qs.extra(select={'casted_code': cast}).order_by('-casted_code')
        return qs
[1]A more complex example can be found here: http://stackoverflow.com/a/26055535/939007

Posted by wu at 10:07 | Comments (2) | Trackbacks (0)
Prev  1   2   [3]