Entries : Category [ Plone ]
Plone is a ready-to-run content management system that is built on the powerful and free Zope application server. Plone is easy to set up, extremely flexible, and provides you with a system for managing web content that is ideal for project groups, communities, web sites, extranets and intranets.
[OpenBSD]  [BSD]  [FreeBSD]  [Linux]  [Security]  [Python]  [Zope]  [Daily]  [e-shell]  [Hacks]  [PostgreSQL]  [OSX]  [Nintendo DS]  [enlightenment]  [Apache]  [Nintendo Wii]  [Django]  [Music]  [Plone]  [Varnish]  [Lugo]  [Sendmail]  [europython]  [Cherokee]  [self]  [Nature]  [Hiking]  [uwsgi]  [nginx]  [cycling]  [Networking]  [DNS] 

29 agosto
2008

setuptools and plone buildout

you need the right version these days...

I've found the same problem 2 times this week, so I think it's worth to write a note about it.

If you are used to work with plone, you probably already know about buildout (or zc.buildout). It is the preferred way for plone-based website development. I'll not go inside buildout itself (just take a look at Martin Aspeli's great book, Professional Plone Development to learn more about it) but I think this could be a useful reminder for those of you who are using it.

In order to use buildout, you need to have some deps installed on your system, one of those is setuptools. As you will need to use python 2.4 with plone (no 2.5 support yet), you will need to install the 2.4 egg for setuptools. Following the installation instructions you will have to download this and install it like:

sudo sh setuptools-0.6c8-py2.4.egg

After that you can install the rest of the buildout deps (like ZopeSkel) using easy_install-2.4. If you do that, you will find a problem when running ./bin/buildout, something similar to:

Develop: '/mnt/data/xxxxxx/zzzzzz.net/plone/zzzzzz/src/zzzzzz.policy'
unrecognized .svn/entries format; skipping .
Traceback (most recent call last):
  File "/tmp/tmpCvdzgq", line 11, in ?
    execfile('/mnt/data/xxxxxx/zzzzzz.net/plone/zzzzzz/src/zzzzzz.policy/setup.py')
  File "/mnt/data/xxxxxx/zzzzzz.net/plone/zzzzzz/src/zzzzzz.policy/setup.py", line 30, in ?
    entry_points="""
  File "/usr/local/lib/python2.4/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.4/distutils/dist.py", line 946, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/develop.py", line 27, in run
    self.install_for_development()
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/develop.py", line 85, in install_for_development
    self.run_command('egg_info')
  File "/usr/local/lib/python2.4/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command
  cmd_obj.run()
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/egg_info.py", line 171, in run
    self.find_sources()
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/egg_info.py", line 252, in find_sources
    mm.run()
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/egg_info.py", line 306, in run
    self.add_defaults()
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/egg_info.py", line 333, in add_defaults
    rcfiles = list(walk_revctrl())
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/sdist.py", line 45, in walk_revctrl
    for item in ep.load()(dirname):
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/sdist.py", line 52, in _default_revctrl
    for path in finder(dirname,path):
  File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/sdist.py", line 98, in entries_finder
    log.warn("unrecognized .svn/entries format in %s", dirname)
NameError: global name 'log' is not defined
While:
  Installing.
  Processing develop directory '/mnt/data/xxxxxx/zzzzzz.net/plone/zzzzzz/src/zzzzzz.policy'.

An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/usr/home/borja/xxxxxx/plone/zzzzzz/eggs/zc.buildout-1.1.1-py2.4.egg/zc/buildout/buildout.py", line 1477, in main
  File "/usr/home/borja/xxxxxx/plone/zzzzzz/eggs/zc.buildout-1.1.1-py2.4.egg/zc/buildout/buildout.py", line 324, in install
  File "/usr/home/borja/xxxxxx/plone/zzzzzz/eggs/zc.buildout-1.1.1-py2.4.egg/zc/buildout/buildout.py", line 556, in _develop
  File "/usr/home/borja/xxxxxx/plone/zzzzzz/eggs/zc.buildout-1.1.1-py2.4.egg/zc/buildout/easy_install.py", line 866, in develop
AssertionError

(notice that the error appears as I've added some modules developed here for my project, like that policy product, src/zzzzzz.policy)

The key there is:

File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c8-py2.4.egg/setuptools/command/sdist.py", line 98, in entries_finder
  log.warn("unrecognized .svn/entries format in %s", dirname)
NameError: global name 'log' is not defined

That error will appear if you are using subversion 1.5 to manage your source code, as the release version of setuptools have some problems with that. To solve that problem, just upgrade setuptools using setuptools (WTF?):

sudo easy_install-2.4 setuptools==dev06

And you are done! (you will have to re-run python bootstrap.py and ./bin/buildout again).

Posted by wu at 18:07 | Comments (0) | Trackbacks (0)
04 noviembre
2008

Plone World Day 2008

Wanna learn about plone? you are welcome...

Plone World Day 2008

During next friday, november the 7th, people from different places all over the world will be celebrating the Plone World Day, an event to promote Plone, share knowledge and get involved with the project.

Talking with some mates at work yesterday, we decided to organize a meeting here in Lugo. Quickly we set up a shedule and we added the event to plone.org. I also created a page in my wiki site (PloneWorldDay2008) with more information about the meeting.

There will be some talks and discussions about Plone, and some time to have some beers afterwards.

This afternoon the pre-inscription form will be avaliable (we have limited room, so we will ask people who like to attend to register themselves before friday) so, if you are interested in participate, just stay tuned!

UPDATE Now you can subscribe yourself in order to attend the event, use the subscription form to do it. It is very important that you subscribe yourself in order to attend the event!

Posted by wu at 11:26 | Comments (0) | Trackbacks (0)
14 noviembre
2008

Plone, GetPaid and the euro symbol

or replacing the currency symbol in getpaid

Plone is currently our CMS-of-choice for almost every project. It was a matter of time that one of our customers finally asks us for an online-shop website. Well, what now? If you do some research mostly everyone is using OSCommerce, Zencart or more recently Magento. Of course everyone will have different oppinions here, the old-and-monolitic oscommerce, it's descendant zencart, the new web-2.0 magento... and there are even some more projects out there.

For me the problem is that all of them are built on top of PHP (which is not a real problem anyway, but...) and they will not integrate so smoothly with my existing production deployment (which is a real problem, that is, adding some more stuff to maintain in the production servers).

So what should I do? I use getpaid. Getpaid is an addon product for Plone that integrates easily on top of the CMS and allow admins to mark any content (really __any__ content!) as buyable/shippable/donatable and more. This is an incredible powerful approach to e-commerce, as you do not have limitation when adding products to your shop. You can use the powerful content types management behind plone and then mark your contents as buyable (a page with a product description and pictures of it, a folder full of pages or whatever else you would like to sell).

After marking that content as buyable, getpaid transparently adds some information to the content, so users could add it to the shipping cart, buy it, etc.

Of course getpaid is a development product, that is, it is not a finished product and it lacks of some features, and that's the reason for that post.

One of those features I missed while setting up a new e-commerce site is the hability to change the shop currency (from USD to EUR for example) as well as the fact that you can't have multiple currencies at the same time (as far as I know).

So, what if you need to change the USD symbol ($) and use the EUR symbol (€) instead?

Well, it is pretty easy:

Of course, this is a quick-and-dirty hack. As proper currency configuration (and probably conversion tools) will be added sooner to getpaid (or I hope so!).

Posted by wu at 18:34 | Comments (0) | Trackbacks (0)
15 noviembre
2008

A week since the WPD

or how to get myself so busy I couln't post about it

A week has passed since the World Plone Day and I've been so busy I couldn't write a line about it.

I've to say it was a very nice experience. At first I thought we will be 2-3 people only, as we arranged everything a little bit late (only 2-3 days before the WPD itself) and we didn't publish too much info about it. But, in the end, we were almost 10 people (with 2 of them connected online through an openmeetings live session.

We spent almost 3 hours with the talks, first covering Plone as a CMS and then Plone as a development framework. During the latter, I'll teach them how to create a buildout environment and how to do create some products on top of plone. Obviously, we didn't have the time to get too far, but it was worth of it.

Then we moved to a nearby bar and we got some beers, keeping ourselves talking about plone, web development and some other topics.

It was a really nice experience, and we talk about the posibility to organize regular meetings, not only plone-related, but related to other web development topics too.

Let's see what happens in the nearby future.

Posted by wu at 11:43 | Comments (0) | Trackbacks (0)
11 febrero
2009

Books: Practical Plone 3

can't wait to get the paperback copy!

Practical Plone 3: A Beginner's Guide to Building Powerful Websites

Some days ago a new book about Plone was published. Practical Plone 3: A Beginner's Guide to Building Powerful Websites is some kind of a community book, as it was not written by one person only. A group of Plone experts has combined their efforts to build such a great work.

This book covers Plone from the basic tasks (create a plone site, add some content to it, manage portlets) to more complex ones (like creating a buildout environment, managing workflows, creating your own content types, etc).

I just ordered my copy some hours ago from Packt Publishing and I'll receive my paperback copy in 6-9 days. Meanwhile I'll be taking a look over the ebook copy (just +5 eur if you order both copies at the same time).

Professional Plone Development

It seems like this new book will be the perfect companion for Professional Plone Development the book from Martin Aspeli that has been by my side since last summer, helping me a lot understanding Plone internals and how to do custom Plone development.

If you are new to Plone and you would like to use it as your de-facto CMS system, let me recommend you to get both books, then you should begin with Practical Plone 3: A Beginner's Guide to Building Powerful Websites and then follow with Professional Plone Development. You will master Plone development in no-time!

Posted by wu at 09:44 | Comments (0) | Trackbacks (0)
13 febrero
2009

A static index page, a plone site and apache

or how to add a splash screen to your plone site

IMPORTANT: This is probably one of the most dirty hacks than could appear in your thoughts when thinking about implementing a plone site! Use at your own shame and risk!!!

Ok, after such an advice, let me introduce you to my problem. For one of our latest Plone sites, we had a very ugly requirement. The site must have a splash screen that should be shown to the user on first load, and each time the user tries to reach the site. That is, the splash will appear when I open whateverdomain.com in my browser, then I'll be able to follow the splash screen links to navigate to the plone site itself and I'll be able to keep navigating through the site as usual. If I (for some strange reason) try to get to whateverdomain.com/news directly, instead of the news, I'll get the splash screen.

Well, I didn't like the idea too much, but if that's what the customer wants, that's what the customer will get.

After setting up the Plone site and Apache, as usual, I ended with a VirtualHost configuration pretty similar to this one:

<VirtualHost *:80>

      ServerAdmin admin@server.com
      ServerName whateverdomain.com
      ServerAlias www.whateverdomain.com
      DocumentRoot /usr/local/www/apache22/data/whateverdomain.com

      ErrorLog /var/log/apache22/whateverdomain.com-error.log
      CustomLog /var/log/apache22/whateverdomain.com-access.log combined

      <IfModule mod_rewrite.c>
          RewriteEngine On
          # prevent access to ZMI through apache
          RewriteRule ^/(.*)/manage(.*) \
              http://whateverdomain.com
          RewriteRule ^/manage(.*) \
              http://whateverdomain.com
          RewriteRule ^/(.*) \
              http://localhost:8089/VirtualHostBase/http/%{SERVER_NAME}:80/whateverdomain/VirtualHostRoot/$1 [L,P]

</VirtualHost>

Then I just saved the index.html file containing the splash screen inside the VirtualHost DocumentRoot (/usr/local/www/apache22/data/whateverdomain.com), and then I tweaked the VirtualHost configuration a little bit, adding some RewriteCond options to the mod_rewrite configuration:

RewriteCond %{HTTP_REFERER} ^http://(www\.)?whateverdomain.com.*$
RewriteCond %{REQUEST_URI} !^/index.html
RewriteCond %{REQUEST_URI} !^/splash_img(.*)
RewriteCond %{REQUEST_URI} !^/splash_css(.*)
RewriteRule ^/(.*) \
        http://localhost:8089/VirtualHostBase/http/%{SERVER_NAME}:80/whateverdomain.com/VirtualHostRoot/$1 [L,P]

Here, I've added some conditions to the rewrite rule that will send the request back to Zope, and get the contents from the application server.

First condition:

RewriteCond %{HTTP_REFERER} ^http://(www\.)?whateverdomain.com.*$

will check for the HTTP_REFERER header, checking if it is the current domain name (with or without www). With this rule we ensure that only requests that match the condition will continue to the final RewriteRule (meaning that, more than probably, you are clicking on links on the site).

The next three conditions:

RewriteCond %{REQUEST_URI} !^/index.html
RewriteCond %{REQUEST_URI} !^/splash_img(.*)
RewriteCond %{REQUEST_URI} !^/splash_css(.*)

are used to check if the request contains those strings on its URI, that is, the index.html splash page, links to the images used in the page and css files too. In fact, we could have used something like:

RewriteCond %{REQUEST_URI} !^/splash(.*)

That would be more generic (for directories containing flash files, for example).

A quick apache restart and it worked!. When trying to access any page within the Plone site directly, the splash screen appeared. If I did follow the links on it, everything worked as expected.

But, the problems appeared soon. What about, for example, links sent by plone to recently created users, by mail? Once you register yourself to such Plone site, you get an email with the link to finish the registration process, but if you click on the link (with the apache configuration described above), you will end in the splash screen (with no choice to get to the confirmation page).

To solve this problem, we only have to add some more RewriteCond magic to the VirtualHost configuration:

RewriteCond %{REQUEST_URI} ^/login [OR]
RewriteCond %{REQUEST_URI} ^/logout [OR]
RewriteCond %{REQUEST_URI} ^/logged_out [OR]
RewriteCond %{REQUEST_URI} ^/password(.*)
RewriteCond %{HTTP_REFERER} ^http://(www\.)?whateverdomain.com.*$
RewriteCond %{REQUEST_URI} !^/index.html
RewriteCond %{REQUEST_URI} !^/splash_img(.*)
RewriteCond %{REQUEST_URI} !^/splash_css(.*)
RewriteRule ^/(.*) \
        http://localhost:8089/VirtualHostBase/http/%{SERVER_NAME}:80/whateverdomain.com/VirtualHostRoot/$1 [L,P]

In those lines you should recognize some of the tipical plone links (login, logout, password, etc). The lines with the [OR] flag at the end will be processed secuentially until reaching the HTTP_REFERER check. That means that the request will be sent to Zope if:

As I've said before, I didn't like the Idea of the splash screen too much, but if you are in a hurry and you need to set up something like that, perhaps you will find this tip useful.

Posted by wu at 20:48 | Comments (0) | Trackbacks (0)
15 enero
2010

Upgrade Plone buildout from 3.1.7 to 3.3.3

the old story of upgrades, with some new exciting action features!

Some months ago I set up a very complex (at least for me) buildout environment. This is the production environment for several Plone sites and it looks like the following scheme:

Plone buildout environment in a production server

I'm not going to explain all the details (as that would be a perfect topic for a doc perhaps one day I should write) but you can imagine the basic idea. Apache will accept requests for www.somedomain.com and will handle them with a VirtualHost that will ask Varnish for the info. Varnish will send back the info directly if posible (thnx to the proxy features) or will ask one of the four Zope instances for the info (using round-robin load balancing). The Zope instances will query the ZODB database served by the ZEO instance.

Well, as I was saying, I set this whole thing up back in april, 2009, and I didn't have the time to keep the versions of all the dependencies in there up-to-date. At the time I decided myself I had to upgrade the buildout, it was running Plone 3.1.7 using a buildout.cfg file (a slightly modified version without the [versions] stuff) to set the basic options of the buildout and a deploy.cfg file for the production environment stuff (creating the ZEO and 4 Zope instances, the varnish recipe, etc). One of the benefits of using buildout is that you can keep track of the needed dependencies in an isolated way, just installing everything you need within the buildout directory, you can even set the versions you want to use of certain parts of the buildout and you can leave the decission on which versions to install directly on the buildout machinery. This is very nice, except when it fails.

Let me guide you through all the pain-in-the-ass upgrade process, a marvelous walk from the omfg-why-is-it-broken to the final lets-reinstall-everything step where I found a solution to my problem ;D

Before going any further, I have to say that I wanted to upgrade the buildout but that was not the only reason to rebuild the buildout env, I needed to add a new site to the server, which depends on two new products/eggs developed by our company. This means I had to add the sources to the src/ directory within the buildout and add the needed stuff to the buildout.cfg file (in that example, the pxgo. stuff).

Ok, after adding that to my buildout, I just tried to run ./bin/buildout without internet access and telling it not to install or upgrade to new versions:

./bin/buildout -No -c deploy.cfg

Using deploy.cfg as the config file (that will extends buildout.cfg)

I'm still wondering why, but it crashes, giving me an error when building simplejson. That was weird, because it is supposed not to rebuild any dependencies... but I thought it could be nice if I ran buildout without the -No options, upgrading the buildout.

./bin/buildout -c deploy.cfg

That was even worse, as it tried to install the latest versions of everything, that is, it tried to install Plone 4-alpha, which has a dependency on Zope 2.12.x, which is not compatible with... you can read more about this here:

As a lot of people were telling me in #plone in freenode that it was my fault since the beginning, because I wasn't pinning the versions of the dependencies in my buildout.cfg file (more on this here: http://www.netsight.co.uk/blog/2009/11/17/configuring-buildout-to-prefer-stable-releases).

Ok, so, at any point it became a common practice to set fixed versions in the buildout.cfg file (I missed that one, I should be more careful and do not let so much time to pass by since the last time I read about buildouts).

Then I tried to add some [versions] stuff into the buildout.cfg (the stuff you see in the linked file) and I tried to rebuild the buildout again:

./bin/buildout -c deploy.cfg

Still got some errors due to problems with versions and dependencies. I kept myself doing tests and more tests and changes and more changes to the config file until someone in #plone (thnx esteele) pointed me to the file .installed.cfg. That file save information about how your buildout was configured, including versions of the stuff in there.

Removing the .installed.cfg file helped, but then (after re-running ./bin/buildout again) I got stucked in another error:

More dependencies versioning problems...

At that point I was very dissapointed. First because the buildout isn't supposed to crash so easily, second because it was kinda hard to find useful information (as it is sometimes with specific errors).

Nevertheless, I tried to approach the problem from a different angle, and I started a new buildout from scratch. I updated the ZopeSkel package for Python 2.4 and then I use the paster tool to create the new buildout:

paster create -t plone3_buildout mangosta_buildout_new

It asked me some questions, like the version of plone I would like to use (by default 3.3.2). Then I use the bootstrap to create the buildout script and all the needed stuff:

cd mangosta_buildout_new &amp;&amp; python2.4 bootstrap.py

And finally I build the buildout:

./bin/buildout

Everything went perfectly well ;D

So, once I had the new buildout I edited its default buildout.cfg file so it seemed something like this:

and I ran :/bin/buildout again... and it crashes again! :( with this error:

The error was related to Products.PloneFormGen and the only way I solved it was editing the buildout.cfg file and replacing the Plone version from 3.3.2 to 3.3.3. After that everything worked as expected (probably because Plone 3.3.3 depends on plone.app.locales 3.3.7 and not 3.3.5).

Fine, I was almost there. I added my deploy.cfg file and re-run the buildout:

./bin/buildout -v -c deploy.cfg

OK! It worked, and I was able to start the zeoserver and the four Zope instances!

Finally, there was only one last step, that is, moving all data from the old buildout to this one. This was really easy, After stopping all the instances from both buildouts I just copied the Data.fs and Data.fs.index files from one instance to another:

cp mangosta_buildout/var/filestorage/Data.fs mangosta_buildout/var/filestorage/Data.fs.index mangosta_buildout_new/var/filestorage/

After copying the files, I restarted the new buildout zeo and zope instances, went to the ZMI and used the portal_migrations tool to upgrade the plone sites within the ZODB to match the src version. Once I finished doing that, I checked every website, everything was working smoothly.

Conclusion

I've (re-)learned some things during this travel:

1- Since Plone 3.2 there is a file that contains a list of versions of all the dependencies for a given Plone version. For example, the file for Plone 3.3 is:

This is very useful and you should use it in your buildout.cfg files as you will ensure yourself that you are installing the correct versions for all the dependencies.

2- Because of [1] the buildout sintaxis had changed, so if you are upgrading from 3.1.x to 3.3.x I recommend you to create a new buildout, migrate your data and use the old one as a backup.

3- When you are working with Open Source Software, it is a bad idea to be out-of-the-game for too long. Once you are back, it will take some time to catch up and you could find some disgusting surprises.

4- Open Source Software Communities are great! If you are in trouble, you have the project website, the official docs, the blog entries from people working on the project, the mailing lists and, of course, IRC. In this case, the Plone community is always there to help.

Posted by wu at 18:04 | Comments (0) | Trackbacks (0)
[1]