Translate your Web application with Weblate [Part 2] - Translation Submodule

In the first part I have explained how to install and configure the basic Weblate instance. In this part, the second one, I'll try to go deeper into details and explain final touches that our concept will need before going to production and being used by both Translators and Developers.

Putting the repo in the right place

Integration of our already converted files (PO) is the most important thing at this stage - we need to combine the translation repository with our project repository (Symfony Project). The thing would be easier if we would not be using Git for our Symfony Project but obviously we do. In this case we will have to attach our translations as a git submodule.

Git submodules are not perfect and lot of people say that it's the worst part of Git of all. I have to partialy agree with it that at the first look it seems wierd and the learning curve is pretty steep.

Before we start initializing the submodule I would like to explain some basic stuff about git submodule and it's behaviour. You can omit this part if you feel that you are a git submodule master ;)

git add <git_repo_url> <where_to_put_it> - this command adds the submodule to both .git/modules/<modulename> and .gitmodules file. It has to be done only once when you initialize your submodule.

git submodule update - this command checkouts the submodule to correct commit. It does not pull the newest commits on a particular branch. To give a sample - you work on a main project (we will call it SymfonyProject) with added submodule (TranslationProject). Your SymfonyProject holds an information about the TranslationProject commit with which it is compatible - for example 515416f.

So we have a situation on which master branch of SymfonyProject points to 515416f commit on TranslationProject.
Now if you checkout the TranslationProject to another commit the SymfonyProject will see it as a change (similar to a file change). Now if you want to go back to the correct version (rejecting your changes) you can use git submodule update.

git submodule update --init --recursive - what it does is pretty simple to the command without parameters but in this case if you never had content in the TranslationProject submodule it will be pulled from the server and then checked out to correct version. --recursive applies it to all submodules within the SymfonyProject. This command is recommended to use.

git submodule status - shows you all submodules with their current checkout version. If you don't see anything listed after firing this command probably something went wrong with the submodule or it's not initialized yet.

Now with all required commands explained we can go to the real fun, lets initialize the submodule. First remember to remove the old translation directory because submodule cannot be created inside / on an existing directory:

rm -rf app/Resources/translations
git rm --cached app/Resources/translations

And now we can initialize the submodule:

$ git submodule add git@bitbucket.org:myOrg/symfonyProject.git app/Resources/translations
Cloning into 'app/Resources/translations'...
remote: Counting objects: 57, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 57 (delta 27), reused 0 (delta 0)
Receiving objects: 100% (57/57), 71.76 KiB | 0 bytes/s, done.
Resolving deltas: 100% (27/27), done.
Checking connectivity... done.

Now if you do git submodule status you should see something similar to this:

$ git submodule status
891962366b5ee75a94cd0a910baa567a5361bcc9 app/Resources/translations (heads/master)

OK! Our module is correctly initialized but if you do now git status in the SymfonyProject you will see this:

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	modified:   .gitmodules
	modified:   app/Resources/translations

That is completely correct - I will try to explain what just happend.

  • .gitmodules file was created, inside you will find some git-helpful information about from where the submodule comes from and where does it sits. This file has to be commited only once unless you change something in the submodule (e.g. repo URL). It's content look similar to this:
[submodule "app/Resources/translations"]
	path = app/Resources/translations
	url = git@bitbucket.org:myOrg/symfonyProject.git
  • app/Resources/translations is shown in staged files - don't worry - it doesn't contain the translation files at all - it's only a checkout hash for the submodule. Each time you change the submodule checkout you will have to commit this change to SymfonyProject.

Ok, now we can commit both .gitmodules and the submodule checkout to repo and push it. Now everybody in the team, while first pulling it will have to remove the app/Resources/translations and fire:
git submodule update --init --recursive to initialize the submodule.

Probably in this point you could ask WHY submodules cannot track the newest version of the submodule - my answer is - they can but in our case we want them to hold particular version for particular branch / commit on SymfonyProject to avoid desync problems. For more info I suggest man git-submodule or git webpage.

Some of you might be interested in submodule creation script which can be useful when migrating, here it is:

#!/bin/sh

set -e
git config -f .gitmodules --get-regexp '^submodule\..*\.path$' |
while read path_key path
  do
    url_key=$(echo $path_key | sed 's/\.path/.url/')
    url=$(git config -f .gitmodules --get "$url_key")
    git submodule add $url $path
  done

It simply parse the .gitmodules file and basing on it's data creates the submodules in right places - very helpful in many cases.

Integration with Symfony

In this stage we have translation repository attached as a git submodule in our SymfonyProject. Fortunately Symfony supports .po files automatically when it recognize the .po extension. In our case all .yml files disappeard after we removed the app/Resources/translations directory and now it only holds PO files. In many cases you will have to refresh the cache to see the current changes - use php app/console clear:cache.

In this point SymfonyProject should use correctly PO files from TranslationProject and react to their changes.

To pull the newest changes simply go to the submodule directory and pull the newest version (or particular checkout hash).

$ cd app/Resources/translations
$ git pull origin master
From git@bitbucket.org:myOrg/symfonyProject.git
 * branch            master     -> FETCH_HEAD
Updating 86797a3..8919623
Fast-forward
 messages.fr.po | 203 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 203 insertions(+)
 create mode 100644 messages.fr.po

Now, if we have the newest changes you have to go back to SymfonyProject and commit the new TranslationProject hash. Seems hard but it isn't:

$ cd ../../../
$ pwd
/home/mgachowski/myProject/
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	modified:   app/Resources/translations (new commits)
$ git add app/Resources/translations
$ git commit -m "Translation Submodule updated hash (French messages updated)"
$ git push

That is all! In the first look may look pretty scary but if you do it 3-4 times you will see that its easy to remember.

Whats next?

For the sake of readability I will split this article into more parts than I though in the beginning.
We have already did the installation and initialization of the repo as git submodule inside our main project. In the next part I will talk about Google Gmail configuration on Weblate and how to setup the production environment for both devs and translators.