FallenRayne Dev One developer's random thoughts are better than nothing.

13Feb/110

Upgrading

So this weekend I decided to upgrade my environment a little bit. I went ahead and purchased a Virtual Private Server with Dreamhost and upgraded my Github account to the Micro plan.

My current project, which I have yet to disclose, needed more room and a repository. I could have just went with SVN on my Dreamhost server but I opted to go with Github in the end because I can more easily get to the files when I need them. I also want to try and keep all my code in one place (whether private or public). With the Micro plan only being $7 a month, I figured it was a good place to start so I can still keep my current project private but have the code on Github. This is my first interaction with Git and I must say that it isn't too bad at all.

With VPS on Dreamhost I really just needed a MongoDB and I wanted complete control over it, so I basically needed either to host it myself or get a VPS. Hosting myself was not really an option as the only box I have that can host it easily is my old Ubuntu machine and I really don't want to try running MongoDB for many users on a 32bit, 3 ghz system with 1gb of RAM. That just sounds like a recipe for disaster. Dreamhost's VPS are 64bit which is important for MongoDB, and they are scalable really easily (and somewhat expensively). I even started doing some of my development on my CR-48 by going into the shell access and sshing into my VPS. I am not a huge fan of vim but it works in a pinch.

The one big problem I am having right now is that MongoDB won't start up on my VPS and the PHP extension isn't working properly. I currently have a trouble ticket in with Dreamhost to check this out for me. I have my data exported and ready to go to put it into the new DB on my VPS, I just need a working DB. Once that is done I already have Fat Free Framework setup and my code in place and working, so I am almost there. I just have a todo list that is about 15-20 items in length before I can launch this thing and start moving on to other aspects of the project. Hopefully I will be able to have a launch post in 2-3 weeks.

29Jan/113

Playing around with MongoDB

So I have been messing around with MongoDB now for about 2 months on a side project that I am working on. I wanted to give a run down on my thoughts about Mongo and the PHP driver for it.

Installation:

The installation of Mongo was relatively simple, mostly because I just installed it on my Ubuntu machine using the Debian package. Got it installed and running in under 20 minutes and had a interface downloaded and in place in about 10 minutes after that. The PHP driver was a simple as running

1
sudo pecl install mongo

and adding

1
extension=mongo.so

to the php.ini file. All in all the process was extremely simple and there were no major hiccups.

Documentation:

The documentation is pretty clear overall. They have some good quick start tutorials and everything is pretty easy to follow. The documentation for the PHP driver is on php.net and just as you would expect PHP documentation, not perfect but most definitely not the worst out there. There have been enough sites that explain usage though I have had some troubles here and there, but I will explain those later.

Ease of Use:

This is where things get fun. Mongo is an extremely fun database to use. The logic of how the documents are constructed just makes so much sense to me as they are essentially just JSON strings. My major passion in web development is JavaScript so this just makes things fun. I am trying to do some great stuff with the queries because I am doing live filtering on the data as the user types. I currently have four different combo boxes and two text fields that the user can type into and automatically filter the content that is displayed. I am able to do this with Mongo's great query capabilities including its Regex search abilities.

The Regex search ability is probably my favorite so far. It has helped me figure out some of the more difficult problems and is extremely fast. I am live filtering on about 12,000 documents or so and checking for values that could be anywhere within a single property of the document and I am getting results back within a second. Regex has also allowed me to get around a missing feature of Mongo that is sorely missed and is probably my only issue with the database; there is no $and operator for the queries. You can do $or, $all, $in, $exists, $ne (not equals), and a bunch of other operators, but no $and. Here is a quick scenario to show why this is missed:

Here are three fake documents. Each one has an id and a tag. The tag just contains a few letters and the user is allowed to enter letters to try and find out if they are in the tag string. The user is expecting only the documents that have ALL of the letters in the tag string to come back, and the letter do not have to be in any particular order.

1
2
3
4
5
6
7
8
9
10
11
12
  {
    _id:'Doc1',
    tag:'ABCDE'
  }
  {
    _id:'Doc2',
    tag:'CBDAE'
  }
  {
    _id:'Doc3',
    tag:'BDAE'
  }

So the user enters in the values of B and C. Doc1 and Doc2 should be returned by not Doc3 because it is missing the letter C. Here is how I would want to create this query:

1
2
3
  $query1 = new MongoRegex('/B/i');
  $query2 = new MongoRegex('/C/i');
  $collection->find('$and'=>array('tag'=>$query1, 'tag'=>$query2));

I am using MongoRegex to create a Regex object that Mongo will understand. Both Regex objects are looking for a single character, and with the $and operator in the query, both regex queries have to return true to return the document. Now this is a very simple example but you can see why it would be useful to have to. Instead I had to resort to pure Regex. Here is the solution I ended up putting in place to handle this exact problem:

1
2
3
4
5
6
  $lettersAry = array();
  foreach($letters as $letter){
    $lettersAry[] = "(?=.*$letter)";
  }
  $regex - '/^'.implode('',$lettersAry).'.*$/i';
  $collection->find('tag'=>new MongoRegex($regex));

That works perfectly for what I need and it puts all of the value checks into one regex object and makes it very easy to pass in as many letters as I want. Is this the perfect solution? Probably not. Does it do the trick? Most definitely.

Summary

Ok, so I didn't have too much to say I guess. I am still working on my app and I don't want to talk about it too much until it is closer to being finished. Hopefully I will be wrapping things up in the new few weeks and I will be able to share more. I will say that MongoDB is very fast, and is very easy to use. I have been able to just focus on my app and less on the database because of this. I haven't had to map out my entire database before I even touched a line of code, instead my database has morphed as I have worked on it. Need to add a new field? Oh well, throw it into the code and it is added! Yay! As a developer working on a side project I really don't want to have to think about the database more than I have too. I just want a place to dump my data and make it easy and fast to retrieve it. Thank you MongoDB for be my data dumping ground.

**Update** Ok, so I have re-written my code to use Kristina's suggestion and I got it working. Not sure why that didn't work for me the first few times I tried but I must have had something all screwy.

Tagged as: 3 Comments
28Dec/100

Phing – An example build script menu system

Got a little delayed on this post due to trying to find a decent code formatter for WordPress.  So here is my example menu script that I built with Phing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
<?xml version="1.0" encoding="UTF-8"?>

<project name="test" default="main" description="A test build script to see how things work.">    
    <property name="choice" value=""/>

    <target name="main">
        <!-- Main Menu -->
        <echo>Welcome to the Phing Build System</echo>
        <echo>---------------------------------</echo>
        <echo>1) Build Scripts</echo>
        <echo>2) Misc. Scripts</echo>
        <echo>x) Exit</echo>
        <echo>---------------------------------</echo>
       
        <input propertyName="choice" validargs="1,2,x"/>
       
        <if>
            <equals arg1="${choice}" arg2="1" />
            <then>
                <phingcall target="buildscripts"/>
            </then>
           
            <elseif>
                <equals arg1="${choice}" arg2="2" />
                <then>
                    <phingcall target="miscscripts"/>
                </then>
            </elseif>
           
            <elseif>
                <equals arg1="${choice}" arg2="x" />
                <then>
                    <echo>Thanks for using the Phing Build System!</echo>
                    <php expression="exit();"/>
                </then>
            </elseif>
        </if>
    </target>
   
    <target name="buildscripts">
        <!-- Build Script Menu -->
        <echo>Build Scripts</echo>
        <echo>---------------------------------</echo>
        <echo>1) Normal Release</echo>
        <echo>2) Bug Release</echo>
        <echo>x) Back</echo>
        <echo>---------------------------------</echo>
       
        <input propertyName="choice" validargs="1,2,x"/>
       
        <if>
            <equals arg1="${choice}" arg2="1" />
            <then>
                <phing phingfile="normal_release_build_script.xml" inheritRefs="true" target="main" />
                <phingcall target="main"/>
            </then>
           
            <elseif>
                <equals arg1="${choice}" arg2="2" />
                <then>
                    <echo>This should call the bug release build script.</echo>    
                    <echo>That script should house what is needed to execute a build.</echo>
                    <phingcall target="main"/>
                </then>
            </elseif>
           
            <elseif>
                <equals arg1="${choice}" arg2="x" />
                <then>
                    <phingcall target="main"/>
                </then>
            </elseif>
        </if>
    </target>
   
    <target name="miscscripts">
        <!-- Misc Script Menu -->
        <echo>Misc Scripts</echo>
        <echo>---------------------------------</echo>
        <echo>1) Do something</echo>
        <echo>2) Do something else</echo>
        <echo>x) Back</echo>
        <echo>---------------------------------</echo>
       
        <input propertyName="choice" validargs="1,2,x"/>
       
        <if>
            <equals arg1="${choice}" arg2="1" />
            <then>
                <echo>This should do something really cool.</echo>    
                <phingcall target="main"/>
            </then>
           
            <elseif>
                <equals arg1="${choice}" arg2="2" />
                <then>
                    <echo>This should do something cooler than the last thing.</echo>    
                    <phingcall target="main"/>
                </then>
            </elseif>
           
            <elseif>
                <equals arg1="${choice}" arg2="x" />
                <then>
                    <phingcall target="main"/>
                </then>
            </elseif>
        </if>
    </target>
</project>

So, here is a quick explanation of what is going on here. First off, the Project tag sets up the project and defines with the default target will be. A target is basically a function call. I then instantiate a property called "choice". This will hold the menu option that is selected by the user. Target main, the default target, is the main menu. You can see the echo tasks that display the menu. The input task asks the user for a value and that value has to be 1,2, or x. The if statement looks at the selected value and each value leads to either a new menu, a task, or exits. For example, if 1 is chosen then the task phingcall will call for the buildscripts target to execute. This will bring up the menu for buildscripts. If 1 is chosen in buildscripts then first it will call for a different build script file, execute that, and then it will return back to the main target with a phingcall task.

The overall goal of this script is to keep the menu up all day long and just have all of the common tasks built into this. I find this very handy on my Ubuntu box just because I always forget where all of the files I need are located and what commands I need to run for certain tasks. If I use a menu system in Phing then I don't have to remember any of that and I can automate some of my more tedious tasks (like database backups).

If there is a better way to set this up or if you have any questions about this script please drop me a line or leave a comment. I am always looking for ways to improve my processes and products.

Hope this helps someone out! Next week I will probably be just talking about my new phone (I get a brand new EVO 4g on Saturday!) and the apps/customizations that will be done. I especially want to play around with Tasker, that app looks amazing!

Everyone have a happy New Year!

22Dec/100

Phing – A quick intro

Phing is a build system that is entirely built using PHP and is based off of Apache Ant. I have been checking Phing out a bit lately due to needing to automate a bunch of tasks at my job. I kept seeing Phing mentioned on various websites and decided that it could be a pretty good choice after seeing what it was capable of. The tasks we want to automate are:

  • Transferring data from one database to another regularly
  • Doing weekly releases, sending e-mail upon completion to our users and the help desk
  • Running PHPUnit tests on a regular basis
  • Building PHPDoc for every release
  • Bringing on new customers (database changes, file changes, account creation, etc...)
  • Hiring new developers (account creation, directory creation, svn checkouts, etc...)

As you can see, we have a decent little laundry list of things that we could automate and with some research I figured out how to do all of these with Phing. The big selling point for us was the fact that Phing can be extended using Tasks. Tasks are basically PHP scripts that are run in Phing through XML tags. This makes anything that is possible for PHP to do for Phing to automate. Phing can send e-mail, manipulate the file system, access APIs, make database changes, and even build a menu system so that all of your common tasks can be accessed from one script even if the scripts are separated into multiple files.

Next week I am going to show an example build script that I worked on that makes a basic menu system that you can leave up all day to accomplish all of your common command line tasks. I used to have one written in Bash but I seemed to have misplaced it and it only ran on Linux anyways. With Phing I can run the build script on Windows, Linux, or Mac.  Better yet, I can even run it on my Ubuntu machine and have it SSH into my Windows machine to accomplish additional tasks if need be.