In June of 2009, mobalean release a patch for the WURFL containing Japanese handsets. This patch ignored the existing data on Japanese handsets in the WURFL as it was sparse and mostly incorrect. However, this also made it incompatible with the core WURFL, and applying the patch either required using one of the more lenient libraries, or manually editing the core WURFL.

After half a year, thanks to the hard work of the WURFL maintainers, these incompatibilities have been reconciled, and the data merged into the core WURFL. Any developer using the WURFL can now use it to start building Japanese sites right away.

Because our patch has been included in the core WURFL, we will no longer maintain a separate patch file. All new handset data will go directly into the core WURFL.

The updated WURFL is available through the WURFL project page.

matsui-san has written a new blog post on ke-tai.org, describing KEITAIALL, a service for viewing technical specs for Japanese mobile phones.  I'm a fan of the site's simple and clean design, that allows you to view information about handsets at a glance, or search for a specific model.

The information provided includes:

• Carrier
• Model Name
• Manufacturer
• Sale Name
• Series
• First Day of Sale
• Browser Version
• Generation (2G/3G)
• Cache
• Screen Width
• Flash Version
• Ranking According to Share

matsui-san ponders about the licensing of this data, and points out that you can view all the data in xml by visiting http://keitaiall.jp/inc/choice.xml.  If developers are free to use and adapt this data as they choose, it could be a great boon to developers.

In the past, a mobile site had to be built using carrier-specific markup to properly function for that carrier. However, now it is possible to use XHTML to build a cross-carrier site, as all Japanese 3G handsets support some form of it [1]. Unfortunately, it is not the same dialect of XHTML: Docomo use i-mode xhtml (based on XHTML mobile profile), au uses XHTML basic, and SoftBank uses XHTML Mobile Profile 1.2. Nevertheless, although there are some differences between these versions, they all share a common base, making it possible to make a site that will function more or less the same across all carriers.

Given that not all handsets support XHTML, if you were to build a mobile site using XHTML, what percentage of handsets would it support? I've scoured the web for the answer to this question, but haven't been able to directly find the answer anywhere. However, as the number of handsets subscribing to 3G vs 2G plans are available, if we assume a handset supports XHTML if and only if it is 3G, we can use these numbers to find the answer.

In total, over 95% of Japanese handsets natively support some form of XHTML. Almost all au handsets support XHTML natively and furthermore the au gateway handles conversion of XHTML to HDML (the markup that older au handsets used). As SoftBank will discontinue its 2G service on March 31, 2010, current subscribers will need to migrate to the 3G service (and handsets that support XHTML). Finally, although Docomo has, relative to the other carriers, a large number of handsets that are not XHTML compatible, Docomo claims that i-mode HTML (which older Docomo handsets use), and i-mode XHTML is mostly compatible. Therefore, I would suggest that if you are building a site for the Japanese market, you do so using XHTML.

[1] Although many Japanese web sites say this is the case, it does not appear to be technically true.  According to Docomo, handsets from the FOMA  2051V, 2002, and 2001series do not support XHTML.  However, we can assume these handsets are few enough to make this generalization.

mobalean is a strong believer in open source.  Collaboration and sharing are at the core of our philosophy.  So we are proud to announce three contributions to the developer community: a WURFL patch file containing about 700 Japanese handsets, a ruby script for parsing the Japanese carrier data and converting it to the WURFL patch file format, and a major update to the ruby WURFL api.

The WURFL is an XML file containing mobile device information such as supported markup types, screen dimensions, and flash lite support.  While the WURFL has a lot of devices in it, including some Japanese ones, the data for them is both poor in quality, and incomplete.  To remedy this, mobalean has created a WURFL patch containing data on all handsets available from the major Japanese carriers (docomo, au, and SoftBank). 

This patch contains data on approximately 700 handsets and has the carrier values for the WURFL capabilities resolution_height, resolution_width, max_image_width, colors, brand_name, model_name, flash_lite_version, xhtml_table_support, and preferred_markup.  While the base WURFL contains additional capabilities for some Japanese handsets, the values of these capabilities are often wrong.  Rather than trying to validate the data in the base WURFL, this patch takes a blank slate approach, and ignores the devices in the base WURFL (with the exception of fallbacks).  All the data in this patch comes from the carrier's official data, and as such is believed to be correct.

mobalean releases this patch to the community in the hope that other members of the community can help improve it.  As with the base WURFL, you are free to use this patch in any manner you so choose.  Our only request is that if you improve the data within, that you also contribute back this data.  Additionally, we hope this patch can eventually be merged back into the core WURFL so that all WURFL users may benefit from it.

To generate this patch, we scraped the carriers' data using a ruby script.  The script transforms the data into an intermediary result, and from that result into a WURFL patch file.  In addition to the WURFL patch file, we have also released our parsing script.  By open sourcing this script, we believe others in the Japanese mobile community, even those who are not using the WURFL, can benefit.  Additionally, we hope that modifications to the WURFL patch be made via this script instead of directly to the patch.  This way, we believe we can more easily keep the patch up to date with new handsets.

In the process of generating the patch, we wanted a way to test the resultant patch file.  We did not want to parse the XML directly, as that would not take in to account the fallback structure.  So we turned to the ruby WURFL API, but found that it did not work out of the box.  As no one else was currently maintaining this API, we decided to take over it.  In doing so, we've turned it into a ruby gem, and have released version 1.1.0 of it.  We hope that this new, easier to install version will encourage further WURFL development within the ruby community.

mobalean hopes these contributions will be useful to other developers.  If you have any questions about these projects, or anything else, don't hesitate to contact us.

• Basic JavaScript support (based ECMA-262)
• External CSS
• Cookies and Referer header
• Copy and Paste within the browser
• Multi-window function
• Increased page size from 100KB to 500KB
• BMP and PNG support

Of particular interest to mobile web developers will be the external CSS and cookie support (rumors about this were mentioned previously).  The lack of these have caused developers headaches, as while SoftBank and au support them, docomo did not.  It will be some time before developers can assume all Japanese mobiles support external CSS and cookies, but it is good to see that we are moving in that direction.

Once more concrete details come out, we'll follow up on this story.

CakePHP is a development framework for PHP that borrows many ideas from Ruby on Rails. I first heard about it when reading through the Merb camp keynote, which states Ruby On Rails is more then 10 times faster then CakePHP (and Merb is even faster). At mobalean, we prefer using Ruby, but if you happen to be using PHP, check out Ktai Library for CakePHP1.2.

The features of this library include:

• Handset detection by user agent (docomo, SoftBank, KDDI, emobile, iPhone)
• Carrier detection by email
• Cross carrier emoji lookup
• Entire page emoji conversion
• accesskey attribute for links
• Display emoji on PC

I had a quick look through the code, and unfortunately while CakePHP is Object Oriented, this code is not. This code, like much of the code for Japanese mobiles make extensive use of a "carrier code" variable. If the library used carrier objects instead, it would be much cleaner. Additionally, there is a lot of duplicated boilerplate code in the KtaiHelper and KtaiComponent class that code be removed in Ruby through the use of the Forwardable module. Nevertheless, if you are using PHP, you might be able to find something that helps you.

(Hat tip: matsuiさん)

emobile has a nice offer here in Japan: a wireless 7.2mbps Internet connection for a bit under 6,000yen a month. We are using the D02HW, a USB HSDPA/HSUPA (3.5G technology) modem, but the following probably also applies to some of their other devices.

To get it to run under Linux, using a modern distribution such as Mandriva, Ubuntu or Slackware, not much configuration is needed. When the device is plugged in, it should be recognized automatically and two serial devices, /dev/ttyUSB0 and /dev/ttyUSB1 are created.

The easiest method to get an Internet connection up and running is to use wvdial (usually part of your distribution). The configuration is relatively easy, you only need to edit /etc/wvdial.conf (as root) and add the following section:

[Dialer em]
Modem Type = USB Modem
Modem = /dev/ttyUSB0
Baud = 460800
Init1 = ATZ
Init2 = ATS0=0
Stupid Mode = on
Phone = *99***1#
Username = em
Password = em

The phone number, username and password are from the device's manual.

Once this is done, you can use "wvdial em" to connect to the Internet.

Tip: if you set the user s-bit on wvdial ("chmod u+s /usr/bin/wvdial" as root), you don't need to be root to successfully create the connection.

A note about emobile and configuration tools which come with your distribution:

• Mandriva will try to configure the device as a "GPRS/Edge/3G" device using comgt, but that won't work correctly because this tool doesn't know emobile (yet, this is with v0.32).
• Other distributions using the GNOME NetworkManager also might not be able to get you connected, at least with Ubuntu it was not successful.

More success / failure stories? Please feel free to post them here!

The value of a default argument in Ruby is the result of an executed expression. Most of the time the value is set to some object:

def int(i=1) i; end
def str(s="") s; end

Under normal usage, the fact that the value can be arbitrary code isn't immediately apparent. However, the value really can be any code:

def foo(a = puts("bar")); end

Furthermore, previous arguments can be referenced in the default argument expression:

def bar(a,b=a) puts "a: #{a} b: #{b}"; end

I wondered how far I could take this, and came up with the following method:

def fib(n, t = n < 2 ? n : fib(n-1) + fib(n-2)) t; end

Although it makes little sense to define methods this way, it does highlight the power of Ruby's default arguments.

If you're using git to manage your homepage / blog / little web application, it is relatively easy to add automatic deployment functionality to simplify your work flow. Deploying a new version can then be a simple git push!

For this article, I'll use a Ruby on Rails application as the example. It should be easy enough to adapt this approach to other situations. You can even do more complex actions such as compiling a deployment archive as long as no manual intervention is required.

Let's assume you want to install your RoR application in /srv/web/my_app, and you have a bare git repository on the same server, in /srv/gitosis/repositories/my_app.git. All you need to do is create a clone from the repository in the deployment location, adjust the permissions of the log and tmp directories, configure the database connection if required and then you are done.

Applying updates with git in such an environment is already simple: develop and test on your local machine, eventually push your changes to the server and finally pull the changes into the deployment directory. For example:

client $ git commit ...
client $ git push
client $ ssh server
server # cd /srv/web/my_app
server # git pull
server # touch tmp/restart.txt

This way, your local configuration is also protected by git and will be merged if possible (commit any local changes).

To further automate this, we can use the post-receive hook. In our example this is located at /srv/gitosis/repositories/my_app.git/hooks/post-receive. I have this hook call a script called update-rails.sh with the application name (my_app) as the parameter.

update-rails.sh looks like this:

#!/bin/sh
name=$1
if [ -z "$name" ] ; then
        echo "need to give name of checkout dir on command line"
        exit 1
fi

dir=/srv/web/$name
if [ ! -d $dir ] ; then
        echo "the directory $dir does not exist"
        exit 1
fi

cd $dir
env -i git pull
rake db:migrate
touch $dir/tmp/restart.txt

As you can see, this script will also run any required DB migrations.

Simple, isn't it?

A bit more on hosting git repositories:

I first tried to simply go with a shared directory on a server, i.e. a directory owned by a specific unix group, the group sticky bit switched on, and all users who are supposed to have access to the repository in that group. This works to some degree, but has some drawbacks as git does not auto-magically handle file permissions correctly. So on a system using a umask of 022 (which I guess is a majority of systems?), newly created files are not writable by the group, only by the user who created the files.

So I decided to go with gitosis, which not only eliminates the file permission problem, but is also a more flexible solution as it doesn't require an actual system user for everyone who is supposed to have access to your git repositories. Instead it relies on your user's ssh keys for authentication plus an ACL to determine read/write access to repositories.

There is already a good setup manual, so I won't repeat everything here. On a server running Debian Etch, it boils down to the following commands (make sure you have the backports repository installed):

# apt-get -t etch-backports install gitosis
# sudo -H -u gitosis gitosis-init < YOUR_PUBLIC_SSH_KEY

That is the setup work required on the server. Then you can clone gitosis' admin repository on your local system:

$ git clone gitosis@YOUR_SERVER_HOSTNAME:gitosis-admin.git

I'll again refer you to the setup manual on how to configure gitosis. Just one thing I stumbled over: When adding new users, make sure that you name the key files correctly. I forgot the .pub extension, and wondered why those users didn't get access...