Amadiere.com

Fourteen and a half crazy frog burpers

25th April 2009

Configuring Subversion Clients

Filed under: Subversion — Tags: , , — Alex Holt @ 3:16 pm

After my new found love of all things Subversion, I’ve been trying to faff about obtaining that perfect config (instead of doing any actual coding!). There are a number of things in the TortoiseSVN settings that I’ve needed to change so far which have caused me a bit of bother maybe. That and I’d like to just point out a few cool things that you can find when digging around TortoiseSVN too.

Firstly, there is the two important settings you should heavily consider before you even start using SVN:

Some important settings for TortoiseSVN

Some important settings for TortoiseSVN

The first thing of note is the big “Global Ignore Pattern” box. This is a space delimited list of files (with wildcards included) that can be used for excluding when you commit to your repository. The original set I left as it was, but added a few key additional files to that list:

*.o *.lo *.la *.al .libs *.so *.so.[0-9]* *.a *.pyc *.pyo *.rej *~ #*# .#* .*.swp .DS_Store *_ReSharper* *.suo Thumbs.db

The key ones you’ll spot I’ve added are *_ReSharper*, *.suo and Thumbs.db. I was keen to avoid putting all the ReSharper files in there by default, and this seemed like a good way of doing this. The *.suo was added when I realised that all I had to do was open the project and this file would change, a little look around shed the light that this file isn’t needed and can be dropped from SVN altogether. Thumbs.db (case sensitivity is important btw), was because I develop on a windows box and it seems intent on creating that damn file everytime I put an image in the directory – so I’ve excluded that too.

The tickbox at the bottom was a little bit more recommended, than something I’ve learnt from. When you have a nosey at the official documentation for TortoiseSVN, they do say that Visual Studio.NET has issues with the dot format SVN directory (Edit: This was only relevant for VS 2003 it seems (see Bert Huijben’s comment below) – so not much of a worry if you are on exclusively newer versions.):

VS.NET when used with web projects can’t handle the .svn folders that Subversion uses to store its internal information. This is not a bug in Subversion. The bug is in VS.NET and the frontpage extensions it uses. Read the section called “Subversion Working Folders” to find out more about this issue.

If you want to change the behaviour of Subversion and TortoiseSVN, you can use this checkbox to set the environment variable which controls this.

You should note that changing this option will not automatically convert existing working copies to use the new admin directory. You will have to do that yourself using a script (See our FAQ) or simply check out a fresh working copy.

While this issue might have gone away (I never experienced it). It seems quite foolish to simply ignore this setting when it does explicitly say against it!

I’ll reserve this post for any future amendments to my standard settings – as much a reference for myself as anyone else.

Wooooo! Time to do some coding!

20th April 2009

MySQL Two-Way, Master-Master Replication

Filed under: MySQL — Tags: , , — Alex Holt @ 1:32 pm

A bit back, I was looking into database replication and specifically, two-way replication. Meaning that either server could be updated and it would replicate any changes to it’s nice neighbour. Microsoft SQL offer a solution – but it was way out of our price-range. So, we looked into the world of MySQL and at the time, the sparkly and new MySQL 5.0.

MySQL offer quite a bit of useful documentation on replication in general, but for our requirement of Master to Master connections, it offered the wonderful barley’s of:

MySQL replication currently does not support any locking protocol between master and slave to guarantee the atomicity of a distributed (cross-server) update. In other words, it is possible for client A to make an update to co-master 1, and in the meantime, before it propagates to co-master 2, client B could make an update to co-master 2 that makes the update of client A work differently than it did on co-master 1. Thus, when the update of client A makes it to co-master 2, it produces tables that are different from what you have on co-master 1, even after all the updates from co-master 2 have also propagated. This means that you should not chain two servers together in a two-way replication relationship unless you are sure that your updates can safely happen in any order, or unless you take care of miss-ordered updates somehow in the client code.

You should also realize that two-way replication actually does not improve performance very much (if at all) as far as updates are concerned. Each server must do the same number of updates, just as you would have a single server do. The only difference is that there is a little less lock contention, because the updates originating on another server are serialized in one slave thread. Even this benefit might be offset by network delays.

This offers some good advice and you really do need to understand exactly what you are doing before you can continue down this path. But suppose that it doesn’t really matter what order things happen, as long as they happen consistently across all servers, what are we to do then to get it all working? Well, enhancing and copying from my response to a question on the awesome Stack Overflow, here is what I’d do.

The first major problem that you must overcome, is that when a new incremental seed is created, it’s normally created 1, 2, 3, 4, 5 (the glorious one times table). However, this will be no good if the theoretical two-inserts-at-once-on-different-boxes happens. It would great two number 4′s. A BIG data-integrity issue. This is solved simply however, by simply providing a seed which equals the total number of masters you would have. So, in my case (and maybe most), there would be two Masters, one would be counting evens, and the other ones odd. 2,4,6,8 and 1,3,5,7. Following this logic there can be no duplicates. It does create the OCD-Offensive side-effect that there are numbers that are never used – but that doesn’t really matter a great deal as far as things go. In addition, you should ensure both your server_id’s are different (two MySQL servers with the same name in replication has distinct disadvantage to the other setups (it doesn’t work) – but that’s standard replication setup anyway. ;0

Master MySQL 1:

auto_increment_increment = 2
auto_increment_offset = 1

Master MySQL 2:

auto_increment_increment = 2
auto_increment_offset = 2

Using all the functionality and commands of the standard MySQL replication, you should then be able to start both servers up as slaves of the other one. Then to check both are working OK, connect to both machines and perform the command SHOW SLAVE STATUS and you should note that bothSlave_IO_Running and Slave_SQL_Running should both say “YES” on each box.

When creating your code to connect to the database servers, you want to send to both boxes equally? Or maybe favour one because it’s a bit beefier? You could add in death-checks to make sure a server is still serving queries and allowing connections – and if its not, use the other one.

All being told, I really like this solution for replication. For the fuller picture, you’d obviously add a slave somewhere that is remotely connected periodically (or all the time) and allowed to update. It acts as an off-site backup should your building be compromised (burns to the ground in a huge inferno-type-mess scenario. For example ;) ).

4th April 2009

The Layout of a Subversion Repository

Filed under: Subversion,Visual Studio — Tags: , , , , — Alex Holt @ 1:07 pm

In and amongst other things recently, I’ve been working on sorting out a replacement for Microsoft SourceSafe / nothing at all. I’ve done my research and I believe Subversion an excellent starting place for using version control and will probably keep most, if not all developers happy. If distributed (and GIT) turns out to be too much of an advantage to ignore – the fact that Subversion is such a mature and well used product, there will always be an upgrade path.

I’ll try not to go into too much detail, as there are plenty of useful resources dotted around that you can find information from. However, before I go onto the directory layout of our repository, I feel I should mention the hardware and software layout of Subversion itself in our environment. Where I work, we are migrating from our historically used Classic ASP code to using C# in ASP.NET, though we have some systems using PHP too. I mention this because we want to be able to use Subversion to look after all code – not just one specific language.

Server Software: In my opinion, I believe you need both Subversion and Apache HTTP for an optimal setup. As we have an Active Directory server as well, it seemed like a good idea to use this rather than create further usernames and passwords for all our developers. For this, you can download and configure everything manually (after all, there is an excellent resource in the free eBook on Subversion) – but actually – you might as well just download the excellent VisualSVN Server. It takes almost all the hardwork out of creating the Subversion server. It’s truely a brilliant piece of kit and if you are new to Subversion, I’d advise using this (or at least having a nosey at it and seeing how it does things – if you’re that way inclined). It’s possibly worth me mentioning that all these applications have the most agreeable price-tag of absolutely nothing at all. Woo!

Client Software: For this, every developer wanting to use Subversion must install TortoiseSVN. TortoiseSVN is a great explorer based client for Subversion that allows developers to perform standard (and advanced) commands from the safety of the right click context menus. This works for all languages and files, but I’d advise for anyone working in Visual Studio, that downloading and paying $50 for VisualSVN is money well spent. It doesn’t require VisualSVN Server, but it gives you a way of utilising all the TortoiseSVN context menus from within Visual Studio, meaning you needn’t worry about mulitple windows and all that. Just concentrate on your coding! Also, because you have installed VisualSVN (or Apache seperately) you can allow (authenticated) users to view the repository in their web browser of choice.

All this and we’re almost up and running. But before we can continue, we need to sort out the structure. This is how I’ve suggested our team uses it:

svn / aspnet  / libraries  / trunk
                           / branches�
                           / tags
              / project01  / trunk
                           / branches
                           / tags
              / project02  / trunk
                           / branches
                           / tags
              / users      / amadiere
                           / user02

Basically speaking:

  • Every language has it’s own repository. There is no need to bring multiple languages into one repository, especially when the amount of code for each language is sufficient to live in its own. I think there may be exceptions to this rule, but those can be taken on a case by case basis.
  • Every language would have a ‘libraries’ directory (this is the equivalent to ‘include’ or whatever). Some things will obviously be used multiple times, so it makes sense to keep them in a central area, this is nothing new. With everything being in one repository however, it means moving something from Project01 to the libraries area will still keep all the history and version attached to it. This is a big plus from where I’m sitting and is why I’ve gone for this layout.
  • Each library or project folder would have the standard 3 directories for trunks, branches and tags. There is plenty of advise and advantages for this structure, most detailed in the SVN book. A project may involve further sub-project folders if it was relevant, before creating the trunk, branches, tags folder structure. However, depending on the nature of the project, it might be better to create multiple top level projects than sub projects with ‘different trunks’.
  • Each user would have their own area (without the 3 standard directory structure). There isn’t a need here for the trunks, branches and tags because this area shouldn’t be used for projects. This area should be used by developers to simply ‘Sandbox’ or test small bits of code etc. There isn’t an end product to need to branch. In the end, this code is the developers to work with – so I guess if they wanted the branches, then they can go for it.

I think with this format, we’ve got a good standing point for where we want to be going with Version Control and the ASP.NET environment. I’m confident, but until we really start using it in anger – we’ll never know! But the idea is then, when I want to develop something, I open Visual Studio and a blank solution. I then add the projects that I’ll need (e.g. Project01, Libraries and Amadiere) and off I go – I have everything open I need and they are all going to their appropriate places in Subversion! Genious!

« Newer PostsOlder Posts »

Theme designed & built for Amadiere.com by Alex Holt. Powered by WordPress