Saturday, October 10, 2015

Moving iTunes library without using the Media folder

Ok, so this post is really not about development (i.e. this is a developer blog), but I just needed to get this weirdness off my chest.

Recently I had the need to move my iTunes library - all the physical files that is - to a new location on my Windows system. Now, I don't use iTunes Media folder due to the "Keep iTunes Media folder organized" setting, since I don't want iTunes to try and organize my music files. I like to handle that myself. However, all guides I could find about moving the iTunes library made use of the Media folder, so I wanted to find another way. The problem is, though, that in theory there isn't any other way.

With iTunes shut down I started to look into where it stores its library information. In Windows that is somewhere like this: C:\Users\[username]\Music\iTunes
In this folder there is a file called "iTunes Library.itl". The file contains information about all music files, including the full path of each music file. But the library file is in a proprietary format with no easy way of reading. Next to the .itl file is another library file called "iTunes Music Library.xml". This XML file contains the same information about the music files, and since it is XML it can be read into any text editor.

So I tried to do a search and replace to change the paths pointing to the music files to the new location. I saved the file and fired up iTunes. But it just ignored the XML completely and started up with an empty library. I then went through all the menus in iTunes to see if I could find a way to do an import of the XML file. No such luck. So I did some more research on the Internet and finally found a post, which after reading I really had my doubts about. But since I couldn't find any other suggestions, I decided to give it a shot.

The procedure is this:

  1. Close iTunes.
  2. Open the XML library file in a text editor and fix all file paths to point to the new location, and then save file.
  3. Open the .itl file in a text editor (it will look all weird) and just remove some of the contents, and then save file. This causes the file to be damaged, which is the intention.
  4. Start up iTunes. It should now notify you that it is reading the XML file, and after a while iTunes will open up with the relocated library loaded.
So it was step 3 that I found a little strange, but hey, doing it actually caused iTunes to read the XML file, reestablishing the moved library. Why, Apple, why oh why?!

As an end note I want to say that I'm not sure if this is always possible to do. In my case it was. But I noticed afterwards that the XML was deleted and didn't seem to be recreated. Maybe it will turn up again at some point.

Update:
Ok, so I just found out something about the XML library file. It is possible to generate it by exporting the library from within iTunes. It's just that finding the correct menu item for this can be a bit tricky. Nowadays, the default setting for the menu bar in iTunes is to not show it. There is, however, a menu at the top left corner in which there is a Library submenu. The problem is, though, that this does not contain anything for exporting the library. Instead you have to select Show Menu Bar, which shows the full menu bar. Then go to File->Library, and this Library submenu does contain a Export Library menu item, which you can use to export the library to XML.
I guess I didn't get the memo when they made that design decision :)

Monday, February 2, 2015

Umbraco deployment using Courier through source control

Disclaimer: I'm being pretty straight forward in this article. However, it is not the intention to endorse or belittle any product or technology, even though there may be statements that could possibly be interpreted that way by someone. Everything in this article is presented as facts as seen from my perspective. And so, if any statements appear to be directly wrong, I hereby apologize. Feel free to leave a comment. 

Introduction

Being fairly new to the world of Umbraco CMS, and coming from the world of Sitecore CMS, I've noticed both some good things and some not so good things about Umbraco.

On the good side, I think it is much easier to get started with Umbraco, than was my experience starting out with Sitecore. On the surface the two may appear similar, but when you actually start doing stuff, I find Sitecore more complex than Umbraco, especially for smaller solutions.

However, when it comes to larger solutions Umbraco definitely has its shortcomings. For example, when doing team development you really start to feel the pain, and it doesn't get better when introducing automatic deployment to multiple environments. Sitecore in itself is not really better in these matters, and it is only by making use of a third party tool, TDS (Team Development for Sitecore), that a real advantage is gained.

With TDS each developer can work in a complete separate environment (his local machine), even including the database. TDS serializes Sitecore items to text files, which can then be part of the solution's source code and therefore checked-in to a source control system. TDS also helps with synchronizing between Sitecore items in DB and textual items in the solution by providing a fairly simple-to-use UI. With regards to deployment TDS can make packages of Sitecore items, which can then be installed on the different environments using a small included command-line tool.

I haven't seen anything at that level for Umbraco. Usually team development occurs on a shared database, but with local source code. This makes feature-driven development kind of hard. Even though you work on isolated source code, you can't make changes in backoffice without the risk of disturbing someone else's work. With regards to deployment Umbraco has Courier, which when reading about it sounds very promising, but which in the current version (2.11) simply doesn't work (to clarify: it fails to transfer revisions to other environments). I hope this will be fixed soon :)

Courier provides the means of creating revisions, i.e. packages of Umbraco items. It lets you select which items to include in the revision, and can even automatically include any dependent items. There is one shortcoming though. When using automatic dependent item inclusion there is no distinction between content items and non-content items. So you'll end up with adding content items to the revision when using this functionality. This presents problems later when deploying to a live environment where editors have created content, since you risk overwriting their work. So it seems better to disable the automatic dependency thingy when creating revisions.

Another shortcoming of Courier is documentation and sample code, which is a couple of years behind the current version (2.11). This is too bad, because you really need this if you want to make use of the Courier API for creating command-line based tools for deployment automation.

Well then, all that being said, I think it's time for what this post is really about - deployment using Courier through source control.

Solution

In the company where I work we have multiple environments: DEV, TEST, PREPROD and PROD. Furthermore, we use GIT for source control, TeamCity as build server, and Octopus for deployment. Ultimately I would like to automate the whole process of deployment, so that I with the press of a button can deploy both application files and Umbraco items. Due to the shortcomings mentioned earlier this is currently wishful thinking. So for now I have settled for a more manual approach. Here's the deal.

In backoffice I have created what I call a long-lived Courier revision. It is just a normal revision, but it will stay there at all times, and when preparing for a release, I will simply just update this revision to reflect the current state of the non-content Umbraco items to be part of deployment. That is, the revision should always contain everything needed to deploy to a fresh environment. And it should be added without auto-including dependent items. Currently in my situation it means to include the following:
  • Datatypes
  • Document types
  • Macros
  • Templates
A note on adding Templates; I'm really only interested in including the database part of the templates, but actually the razor files are also added, which is unnecessary since they will already part of application files deployment. It is not catastrophic, just inconvenient.

Due to the issue mentioned earlier with Courier 2.11 unable to transfer revisions, and due to the fact that we (developers in my company) don't have access to PREPROD and PROD environments, an ingenious scheme had to be devised for getting the revision moved to these environments.

The solution is to include the revision as part of the source code. The revision is simply a folder structure with a bunch of files located at App_Data\courier\revisions\ and adding this to the GIT repository is perfectly doable. Since revision files are just plain XML files, having these source controlled gives the added benefit of being able to inspect changes (git diff) before committing.

Now, when deploying to an environment the revision just follows any other application files, and it is then a simple matter to go into to backoffice on that environment, select Courier, and install the revision.

I find this approach simple and pragmatic. However, I do hope to be able to automate it more in the future when Courier becomes a bit more mature/stable.