» blog.e-shell.org

OSx, core files and disk space

OSx has its roots in the Unix world. It is based on Mach 3.0 and FreeBSD 5, so you could expect some Unix-like behaviour from it.

One of those things that have been laying around in the Unix lair for a loooong time are cores or core dumps. A core is a file that appears when an application crashes, usually with a segmentation fault or some other kind of ugly error. Inside that file, if you use some proper application to examine it, you will find a memory dump, that is, the in-memory space using by the application while it was running.

As I've said, using tools like ddd or gdb you could learn a lot about how and why the application crashes, and that information is very important when doing software development.

The problem with that core files is that they are usually very big, that is, something between 300Mb and 1Gb (depending on the ammount of memory in your computer). That means that when a core appears, a lot of space is wasted (well, at least wasted if you are not a developer debugging some app).

In other systems like FreeBSD, OpenBSD or some Linux distributions, those core files appeared inside the directory from which the application is called, which in most cases is the home of the user running the application. That's fine, because you will notice those big files and you can delete them whenever you want.

In OSx the core files are saved somewhere else, in a someway hidden directory, called /cores (well, not completely hidden, but difficult to find for non-expert users). That was my case 2 days ago, while doing some housekeeing work in my Macbook, I found that directory wasting around 1Gb of disk space:

snowball:/ Wu$ sudo du -sh *

[ ... ]

1.0G    cores

[ ... ]

snowball:/ Wu$ ls -l cores/
total 2110848
-r--------   1 Wu  admin  152924160 Feb 27  2007 core.1216
-r--------   1 Wu  admin  168681472 Feb 21  2007 core.24998
-r--------   1 Wu  admin  168681472 Feb 21  2007 core.25018
-r--------   1 Wu  admin  590467072 Apr 16  2007 core.8464
snowball:/ Wu$

As you probably noticed, the core files are quite older, from February and April, and I do not need them anymore, so I deleted them directly:

snowball:/ Wu$ rm -f cores/*
snowball:/ Wu$

(remember to use the -f switch, because they are protected read-only)

In my case it wasn't a big deal, it was only a matter of 1Gb, but getting more crashes will mean more core files, and the filesystem being filled with some data probably you will never use. So it is a good idea to take a look inside /cores from time to time, and delete the cores you do not need.

The other option is to prevent OSx from creating core files at all, we can achieve that using sysctl:

snowball:~ Wu$ sysctl kern.coredump
kern.coredump = 1
snowball:~ Wu$

That means core dump creation is activated, we can disable it manually:

snowball:~ Wu$ sudo sysctl -w kern.coredump=0
kern.coredump: 1 -> 0
snowball:~ Wu$ sysctl kern.coredump
kern.coredump = 0
snowball:~ Wu$

And then adding one line to /etc/sysctl.conf so dump creation keeps disabled on the next reboot:

kern.coredump=0

OSx dns cache

Three in a row... I think this is the first time I write so much in a day.

This is a quick reminder for myself, but you (OSx users) out there should find it pretty interesting too.

In OSx it is posible that you have suffered the following:

1- You access a site, using your favourite protocol/application (ssh, http, ftp, svn, etc), for example, test.e-shell.org. This name points to ip address 192.168.1.1

2- The ip address for test.e-shell.org have change, now it is 192.168.1.2, but (you have no idea why), your OSx still tries to connect to the old ip address.

3- You add an entry to /etc/hosts, with the new ip address, trying to force the system to get the point, but no luck, it still tries to connect to that old ip address

Well, the reason for that, the culprit, is lookupd (dscacheutil if you are using leopard). This little daemon acts as a DNS cache, caching up all the names and related ip addresses.

So, to get the new ip for test.e-shell.org, we will have to refresh the cache, like that:

lookupd -flushcache

(from a Terminal.app, for example)

Note that in leopard, it should be something like:

dscacheutil -flushcache

From Tiger to Leopard in 10 steps

Almost one year have passed since I bought it, and yesterday I upgraded my MacBook from Tiger (10.4.x) to Leopard (10.5.x). Here is the process, step by step:

1- Get an external hard drive, in my case a 120Gb 2.5" Western Digital laptop-like IDE disk inside a USB 2.0 external case.

2- Get rid of all the unnecesary stuff in my actual macacosx installation (a lot of rm -rf of directories like screencasts, mp3, anime, etc and a lot of scp -r of some other directories like codigo23, e-shell.org or backgrounds).

3- Do whatever specific backups are needed (dumps of PostgreSQL and MySQL databases, exports from Zope, etc)

4- Use Superduper to create an exact, bootable copy of my actual Tiger installation (so, in case of trouble I could boot from the external hard drive, or use Superduper again to get the system fully restored). Important: You really should get a firewire external disk instead of an USB one (as superduper recommends) if you are in a hurry, in my case, even using USB 2.0 it tooks sooooo long, you have been warned.

5- Use the bootdisk option from the system preferences in my actual Tiger installation to boot from the cloned system in the external USB hard drive. Once it boots up, check everything is ok and use the bootdisk option to boot again the Tiger installed in the macbook hard drive.

6- Put the Leopard installation DVD inside the Superdrive and reboot the laptop (do not forget to press the C letter in the keyboard, so it boots from the DVD! (thnx Clayton!).

7- Follow the on-screen instructions (in my case disabling installation of unneeded languages like Japanese or Italian)

8- Wait 30-40 minutes

9- After a reboot fill in the needed information and et voilá! Leopard installed cleanly in the MacBook

10- Finally install all the needed software, like Carbon Emacs, Macports, Firefox, Opera, Burn and a lot of widgets. Of course, put back in there all the data from the backups.

I've to say that the process was totally smooth and unpainful and everything seems to be working just fine. Hope that the fact Leopard is a 64-bit operating system will make the difference...

Leopard desktop

That's the look of my laptop after the Tiger-to-Leopard upgrade:

I've customized the Terminal.app a little bit too (the most used app in the laptop, for sure!)

Some new things in Leopard, the new spaces (something similar to the virtual desktops on most Unix-like system window managers):

That image above is the result of pressing F8 to see all the spaces at once. An interesting thing about spaces is that you can use expose on top of them, to get a whole view of your windows:

Of couse spaces are very simplistic if you compare it with the virtual desktops from enlightenment or some other window managers, but I hope they will work to get more advanced versions in a near future.

Another add-on from Leopard is the use of CoverFlow in the Finder. That effect was used before in iTunes, and now they have put it in the Finder itself, so you can use it to view the contents of any folder.

It will display previews of most filetypes, like contents of PDF files, a thumnail of an image, or the contents of a Python source code file:

There are some other new things inside this new version of macacosx (like Time Machine or Quicklook), you can check them all here.

man paths in OSX

After upgrading to Leopard, I've found a weird thing about it, and after discussing that with some friends, seems like more people is having the same problem.

One of the first packages of software I've installed on the new OSX version is MacPorts (something very similar to those ports collection on FreeBSD or OpenBSD). After installed some packages using that, I noticed that I couldn't accesss the man pages of those packages, nor even I could access port man page itself.

The ugly error I got evertime was something like:

snowball:~ wu$ man port
Cannot open the message catalog "man" for locale "es_ES.UTF-8"
(NLSPATH="<none>")

No manual entry for port
snowball:~ wu$ man psql
Cannot open the message catalog "man" for locale "es_ES.UTF-8"
(NLSPATH="<none>")

No manual entry for psql
snowball:~ wu$

After some digging, I got to the point that there was some problem with the MANPATH environment variable, seems like the MacPorts man path wasn't there:

snowball:~ wu$ manpath
/usr/share/man:/usr/X11/man
snowball:~ wu$

(That wasn't a matter in Tiger)

So, to solve the problem it was as easy as adding two lines to my ~/.profile:

MANPATH="/opt/local/man:${MANPATH}"
export MANPATH

After that, opening new Terminals result in the correct path added and me being able to access those man pages:

snowball:~ wu$ manpath
/opt/local/man:/usr/share/man:/usr/X11/man
snowball:~ wu$

PORT(1)                   BSD General Commands Manual                  PORT(1)

NAME
     port -- operate on individual or multiple Mac ports

That one was easy but, at first, that message about the catalog and the es_ES.UTF8 locale made me think about something harder to find.

Changing GTK themes in OSx

If you, like me, usually find yourself using a lot of open source apps in your OSx system and that applications tend to be GTK based (like dia to draw schemes and diagrams or sylpheed as your MUA), you will see something like that when running that GTK-based application:

Pretty ugly, even more if you like all your applications having the same look. GTK already has some good-looking themes and there are even some themes that mimic the look of OSx itself (just take a look at the gnome-look project to find some.

The problem I found once I downloaded all the themes I would like to check was how could I change the GTK theme within OSX??.

In Gnome there is a tool that let you change every aspect of a Gnome theme, from the GTK theme to the iconset, backgrounds, etc. But obviously, I do not want to install Gnome, just only a way to change easily my default GTK theme.

The answer was right there, in the MacPorts collection:

snowball:~ wu$ port search gtk-theme
gtk-theme-switch               x11/gtk-theme-switch 2.0.4        command-line theme switcher for gtk2
snowball:~ wu$

A command-line tool... could it be better?

Installing it was easy:

sudo port -v install gtk-theme-switch

And, yes, it could be better!, you can use that tool to change the GTK theme from a console, you can use it to install .tgz compressed themes and it has an easy-and-quick GTK-based gui to do the job too. Amazing!

To use the graphical interface, you only have to call switch2 from a Terminal.app, to get something like:

There, you only have to select your favourite theme (if you do not have themes installed, try port search gnome-themes in a shell) and click apply to change the default theme to:

Well, as we are using OSx, it should be nice if our GTK apps look like the rest of the native applications, so just download the theme tarball, and use switch2 to install it, both from a shell or using the graphical interface.

Once you have your theme installed, you will be able to select it from the dropdown menu in the graphical interface, to use it:

Much much better, isn't it?

Trimmit

Reading Alexander Limi's blog two days ago, I found an interesting article about Trimmit. It is a tool that cleans your MacOSx binaries, removing some innecesary stuff to make them smaller than the original size.

Alexander just tried the app himself, with some interesting results, like the Addressbook app ending with a size of 4Mb instead of the 55Mb it is sized by default or Mail.app being resized from 289Mb to 21Mb. The drawbacks are that not all applications could be trimmed, but most of that untrimmable apps are third-party ones (which in my case didn't waste so much space). There's a list where you can check if a given app could be trimmed or not.

That's a good way of get some free space in laptops with smaller hard drives (like my macbook with the 80Gb hard drive).

Unexpected https connections coming out of my laptop

Today, while debugging some local network connections with netstat in my laptop (running OSX 10.8.5) I noticed a very weird thing:

tcp4       0      0  192.168.43.201.56227   17.143.164.144.443     ESTABLISHED

"A connection to https somewhere, while all I'm doing is debug some local stuff in a shell and edit some files in emacs?" - I thought. Something smell badly.

I did a whois on the ip address:

whois 17.143.164.144

And I saw there to whom the address belong:

OrgName:        Apple Inc.
OrgId:          APPLEC-1-Z
Address:        20400 Stevens Creek Blvd., City Center Bldg 3
City:           Cupertino
StateProv:      CA
PostalCode:     95014
Country:        US
RegDate:        2009-12-14
Updated:        2011-03-08
Ref:            http://whois.arin.net/rest/org/APPLEC-1-Z

"Ok, this has to be one of those services running by default on the background by OSx then, let's find out which one" - I told myself.

At first thought I decided to use netstat itself to find out which process was sending such a long-term request (the connection did stay open for quite some time). Too bad OSX's netstat does not have a -p flag to show the processes (or at least not in this OSx version). OSx's fuser cannot be used for that purpose neither, and it does not have FreeBSD's sockstat or fstat tools.

It has lsof though, which is good enough for the job:

sudo lsof -n -i 4 -a|grep 56227

There it was:

apsd        317           root   10u  IPv4 0xf041bb92eb553d51      0t0  TCP 192.168.43.201:56227->17.143.164.144:https (ESTABLISHED)

"apsd? wtf is apsd?" - Question popped in my mind, and all of a sudden the name become familiar - "I'd say we have met before... let's look at the man page":

APSD(8)                   BSD System Manager's Manual                  APSD(8)

NAME
     apsd -- Apple Push Notification service daemon

SYNOPSIS
     apsd

DESCRIPTION
     apsd ApplePushService daemon for Apple Push Notification service.  This
     is part of the ApplePushService framework.

     There are no configuration options to apsd.  Users should not run apsd
     manually.

Mac OS X                         Feb 10, 2009                         Mac OS X

HA!, I loved this part: "Users should not run apsd manually", which we could translate into "Hey you!, do not mess with this stuff that could be sending some data back to us here!".

The man page did not return too much info, but a bit of searching through the Internet for Apple Push Notification service daemon returned a couple of helpful links:

• Some information about that notification service itself: https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Chapters/ApplePushService.html
• An entry in stackoverflow about disabling it: http://apple.stackexchange.com/questions/92214/how-to-disable-apple-push-notification-service-apsd-on-os-x-10-8

Good enough for me (specially the second one). A quick read through the documentation told me that this probably is used by Apple to send me upgrade notifications and things like that, which I don't care about, specially when I'm working from a remote location, connected through my mobile phone and a limited bandwidth line.

So, I did shut it down:

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.apsd.plist

(turn it back on is easy, just replace unload with load there).

Finally, I added this alias to my ~./profile, so I could list processes listening on tcp4 ports quickly:

alias tcp4_procs='sudo lsof -n -i 4 -a'

(sudo is needed in order to see process my user does not own).