Omit unexpected XML Elements with XStream

XStream is a great library to create XML from objects and vice-versa, and one of the many areas I use it is to store configurations. One problem is that XStream has the hability to ignore fields when serializing objects, but not the opposite – e.g., if it finds a tag that does not have a corresponding property in your class, it will throw an exception.

It is not clear to me why they still don’t have anything to handle such situation, which is fairly common to happen. For example, you may have a XML and only needs a small fraction of its data, or maybe you changed the way you store configurations and removed some properties. In both cases you will have to map the entire object graph, even if you don’t want it. Their FAQ page states this:

If a field is removed from the class, deserializing an old version that contains the field will cause an exception. Leaving the field in place but declaring it as transient will avoid the exception, but XStream will not try to deserialize it.

I find this behavior pretty odd and annoying, but luckily there is a workaround: the XStream class has a protected method named wrapMapper(MapperWrapper next) that subclasses may implement in order to tell the library if a given class or property should be considered. In this method you can add verifications for the fields or classes that you don’t want to bother, and return false for them.

Check out a working example:

XStream x = new XStream() {
	@Override
	protected MapperWrapper wrapMapper(MapperWrapper next) {
		return new MapperWrapper(next) {
			@Override
			@SuppressWarnings("rawtypes")
			public boolean shouldSerializeMember(Class definedIn, String fieldName) {
				if (fieldName.equals("shouldCopyWithProject")) {
					return false;
				}

				return super.shouldSerializeMember(definedIn, fieldName);
			}
		};
	}
};

In the previous example, I had a field named “shouldCopyWithProject” that was removed in newer versions of my application, but such refactoring resulted in crashes when opening the files from older versions of the app, so I had to handle the situation manually.

It would be great if XStream had a cleaner solution for this, but I think it is very unlikely to happen. Nevertheless, the approach here described works very well.

Update

As Chris have pointed out in the comments, it is much easier to just use the omitField(class, fieldName) method, as in

// .....
XStream x = new XStream();
x.omitField(A.class, "shouldCopyWithProject");
// .....

I came across this method several times, but for some reason I thought it only worked for serialization. Anyway, thanks to Chris for simplifying our lives :)

Easy and fast XML node access with TBXMLEx

I recently added a very nice feature to one of my Open Source projects, TBXMLEx, which is the possibility of directly access any node without having to loop through all its parent nodes. This is very useful when you know beforehand where the node is located at. The sintax is very clean and concise. Suppose you have the following XML:

NSString *xml = @"<data> \
        <a> \
            <a1/> \
            <a1/> \
            <a1/> \
        </a> \
         \
        <b/> \
        <b/> \
        <c/> \
         \
        <d> \
            <d1> \
                <d11/> \
                <d21/> \
            </d1> \
             \
            <d2> \
                <d21/> \
                <d22> \
                    <d221/> \
                </d22> \
            </d2> \
        </d> \
    </data>";

and that you need to access the “d221″ node, which is pretty deed inside the structure. Using TBXMLEx it is straightforward do to so:

TBXMLEx *parser = [TBXMLEx parserWithXML:xml];
NSArray *result = [parser.rootElement query:@"/d/d2/d22/d221"];
TBXMLElementEx *element = [result objectAtIndex:0];

The key point in the previous code sample is the “query” method (a member of TBXMLElementEx), which takes a path as argument and returns all nodes that matches the criteria, as TBXMLElementEx instances. Right now it supports only simple expressions, but nevertheless it’s a quite useful feature .

You may wonder why not use XPath instead. The point is, while very powerful, you’ll need extra libraries to work with XPath, and it can take some time to master its syntax. The “query” method I implemented does not intend to take over XPath nor reimplement its features, it is just a convenient, syntax sugar method to do regular tasks we face on a daily basis, and while someday it may be improved to be more powerful, XPath will always be the choice when you need to do nasty thigns with XML.

TBXMLEx is an Open Source library freely available at https://github.com/rafaelsteil/tbxmlex

Character encoding issues with Jetty 8

After deploying a webapp to the production server, all pages showed character encoding problems (aka, an accented letter would display “strange” data instead). In my development box it worked like a charm, but I couldn’t get it to work in the live server. I had all files saved as UTF-8, as well calls to  ”<%@ page contentType=”text/html;charset=UTF-8″ pageEncoding=”UTF-8″ %>” in JSP files, “<page-encoding>UTF-8</page-encoding>” in WEB.xml etc…, but nevertheless, it simply didn’t work. I even used an encoding filter as suggested by a friend, without any luck.

What actually solved the problem (if I can call it that way) was a Jetty downgrade from version 8 to version 7. Simple as that. I don’t know what changed in version 8, and maybe it is just a matter of changing a setting somewhere, but until I find which one, sticking with Jetty 7 works great.

Easy, reliable and free SMTP with SendGrid

It is a problem as old as the web itself: which server or service to use when we need to send emails from our website or local computer? It may sound an easy answer, but if your hosting provider does not offer an out-of-the-box service with fair usage rules, setup a SMTP server yourself is a task that can get messy really fast – either to setup and to maintain it.

What would you use? sendmail, postfix, postmaster, procmail, qmail? Damn, probably half of these options even aren’t smtp servers. And after you find one that looks good, there are so many configuration options and terms to deal with that you will want to swear. Not to mention spammers trying to flood your server.

So, instead of going through this deep, dark and painful way, I found out that using a third party SMTP service is one of the best decisions one can make. For the last months I have been using SendGrid and I am totally happy with it: easy and intuitive panel, simple and clear setup instructions, good prices and – the better – they have a FREE account, which you can use to send up to 200 emails per day. If you think for a while, 200 emails per day is A LOT, if you have a small website or small delivery of emails. It is also perfect when you are developing at your local machine and need to test the email delivering of something (like a registration or recover password form).

To integrate you only need to change three SMTP settings in your application:

  1. SMTP Server: smtp.sendgrid.net
  2. Username: your account username (the same you use to login in the website)
  3. Password: your account password (the same for the website)
As usual, the SMTP port is 25.
Besides that, SendGrid.net even has a REST API and plenty of documentation. So, if you are looking for a reliable, easy to use and hassle-free SMTP service, check out SendGrid

Correctly loading resources (images and files) in Java

To load a file (be it a text file or image) in Java is fairly easy, and most of the time all you need if the path to the resource. However, the filename approach does not work so well when your data is inside a JAR file. JAR files are basically ZIP with a different extension, and a regular filename don’t make sense when the resources are inside the jar.

This is a pretty common problem that many developers face when their app goes into production (including Swing apps), because when they are developing all files are exploded (e.g., not inside a jar), and thus loading images and files using paths like the example below work fine

// This will work when your app and resources
// are not contained inside a jar
String path = getClass().getResource("/path/to/image.jpg").getFile();
Image image = new ImageIcon(path).getImage();

Note the call to getFile(). However, getFile() will return invalid paths once you pack your application, resulting in errors.

In order to make it work right, you should use the getResource() method of Class (or getResourceAsStream()) without the call to getFile(), and instead use the URL or Stream directly. The previous example will then be converted to

// ImageIcon, like many others, can receive an URL as argument,
// which is a better approach to load resources that are
// contained in the classpath
URL url = getClass().getResource("/path/to/image.jpg");
Image image = new ImageIcon(url).getImage();

The very same approach works for files, but instead you’ll use getResourceAsStream().

Very useful reflection library for Java

Reflection in Java is a subject as old as Java itself, and likewise is the kind of thing that you can easily forget the details after some time without dealing with it. For example, to get a field from a given class, a call to getClass().getField(fieldName) will do the job, but only if the field is not private, which then you should use getDeclaredField() and call setAccessible(true). But again, only if such field is declared in the same class you are querying – e.g, it does not apply to the super class.

The list goes on and on.

There are many libraries out there, from the very low level to the high level. One good example of the latter is ReflectUtils, a very convenient and easy utility class.

Sample code:

    • Getting a value from an object field
Object value = ReflectUtils.getInstance().getFieldValue(someInstance, "fieldName");
    • Setting a value on an object field
TestEntity thing = new TestEntity();
ReflectUtils.getInstance().setFieldValue(thing, "entityId", 33);
// value of thing.getEntityId() will be "33", value is autoconverted into the right type
    • Setting a nested value on an object field
Object thing = new HashMap(); // using a hashmap for simplicity here, could easily be nested POJOs
ReflectUtils.getInstance().setFieldValue(thing, "person.contactInfo.name", "aaronz");
// the value of the name field which is on the object in the contactInfo field which is on the object in the person field on the thing object is set to "aaronz"

The library is available at http://code.google.com/p/reflectutils/

 

Tip – Formatting a FAT32 Hard Drive of up to 2 TB in Windows 7

To have to format an huge hard drive using FAT32 as filesystem is a quite unusual requirement, unless you own a Sony Blu-Ray player (and possible others). While my TV is able to read a NTFS filesystem just fine, it does not reproduce DTS audio, and reencoding the video can take hours depending of your source file. Also, sometimes the TV (an LG 47″ Full HD TimeMachine model) drops frames if the video is very high-res (say, a 1920×1080 .MKV file with an high bitrate), which leds me to use the Blu-Ray player.

The only problem is that it does not read NTFS, only FAT32, so I had to convert my 1 TB hard drive first. In order to do that in Windows 7 (and possible other versions of Windows), it is necessary to appeal to third party tools due to the HD’s size. One of the simplest and fastest that I have found is fat32format, which does a pretty good job, handling drivers of up to 2 TB.

Note: Please bare in mind that while you can format a hard drive of up to 2 TB, FAT32 has a 4GB limit per file, and there is nothing you can to about it. If your files are bigger than 4GB, you will need to slipt them or move to NTFS.

fa32format is a free tool, available at http://www.ridgecrop.demon.co.uk/index.htm?fat32format.htm

Fast XML parsing in iOS / Objective-C with TBXMLEx

Have you ever wondered which is the best way to parse a XML file in Objective-C / iOS? There are certainly a lot of options available to those who know where to lot at, but many of them are just awful to use: hard to configure, cryptic API, inexistent documentation and so forth.

Ray Wenderlich once wrote a post called “How To Choose The Best XML Parser for Your iPhone Project“, which greatly covers many XML parsers available to iOS, and was there that I discovered TBXML, which is by far one of the best parsers available to iOS. While TBXML is great, it’s “non-OO” API and some intolerance to bad formed XML had me wonder if there were anything I could do to make it better.

With that motivation, I created an extension to TBXML called “TBXMLEx” (TBXML with Extensions) which adds some syntax sugar on top of the original library and better handle bad XML files, all of that with a more OO-friendly interface.

The design goals for TBXMLEx are (taken for the original TBXML goals):

  • XML files conforming to the W3C XML spec 1.0 should be passable
  • XML parsing should incur the fewest possible resources
  • XML parsing should be achieved in the shortest possible time
  • It shall be easy to write programs that utilise TBXML
How to use it
Below is a quick example of how to use TBXMLEx:
#include "TBXMLEx.h"

NSString *xml = @"<files> \
    <file timestamp='1234567890' size='123' createdAt='01/01/20011'>file1.jpg</file> \
    <file timestamp='1234567890' size='8934'> \
        <name>file2.jpg</name> \
        <attributes> \
            <createdAt>01/01/2011 13:45:56</createdAt> \
            <owner>john</owner> \
        </attributes> \
    </file> \
</files>";

TBXMLEx *xml = [TBXMLEx parserWithXML:xml];

// "files" is the rootElement
if (xml.rootElement) {
    TBXMLElementEx *fileNode = 1;

    while ([fileNode next]) {
      // You can access the attributes through a dictionary
        NSDictionary *allAttributes = fileNode.attributes;
        NSLog(@"Timestamp: %@", [allAttributes objectForKey:@"timestamp"]);

        // Or you can have direct access to any specific attribute
        NSLog(@"Size: %d", [fileNode intAttribute:@"size"]);

        NSObject *createdAt = [fileNode attribute:@"createdAt"];
        NSString *filename = fileNode.value; // or fileNode.text

        if (!createdAt || !filename) {
            // Look for the properties someplace else
            TBXMLElementEx *nameNode = [fileNode child:@"name"];

            if (nameNode) {
                filename = nameNode.value;
            }

            TBXMLElementEx *attributesNode = [fileNode child:@"attributes"];

            // If an attribute does not exist it will simply return "nil", but nevertheless it's
            // always good check if it exists if you really need it
            if (attributesNode) {
                NSLog(@"Created at: %@", [attributesNode child:@"createdAt"].value); // Will not crash if attribute is nil
                NSLog(@"Owner: %@", [attributesNode child:@"owner"].value);
            }
        }

        NSLog(@"Filename: %@", filename);
    }
}

You can find more information about the project, as well the source code, at https://github.com/rafaelsteil/tbxmlex

Loading a Haar Classifier with OpenCV in Mac OS X without hassle

OpenCV is a great library, and there are a lot of examples around the Internet, and while many of them date back to the OpenCV 1.x release (which is not compatible with OpenCV 2.x), you can just install and link against the 1.x release without any problem.

On the other hand, you may face some trouble getting the examples to compile and / or run correctly, depending of how you installed OpenCV and the example source code you are looking at. In my case I installed the library in my Mac OS X using homebrew, which installs both the 1.x and 2.x versions, with the includes at /usr/local/include and libraries at /usr/local/lib. Then, just import prefixing <opencv/….> or <opencv2/….>, like

#include <opencv/cv.h>
#include <opencv/highgui.h>

Another problem I faced was when trying to load a Haar Classifier, which should be a very simple operation, but for some reason it fails miserably with the error

OpenCV Error: Unspecified error (The node does not represent a user object (unknown type?)) in cvRead, file /tmp/homebrew-opencv-2.2-mR8t/OpenCV-2.2.0/modules/core/src/persistence.cpp, line 4811
terminate called after throwing an instance of ‘cv::Exception’
what(): /tmp/homebrew-opencv-2.2-mR8t/OpenCV-2.2.0/modules/core/src/persistence.cpp:4811: error: (-2) The node does not represent a user object (unknown type?) in function cvRead

A lot of posts in the forums relate this same problem, and most of the suggested workarounds tell to perform any dummy operation on a IplImage before loading the classifier, like

IplImage* hack = cvLoadImage("dummy.jpg", CV_LOAD_IMAGE_COLOR);
cvErode(hack, hack, 0, 3);
cvReleaseImage(&hack);

but that didn’t work for me (it may work for you, though… just give it a try). What I’ve found out is that I should link against opencv_objdetect as well. One may argue that was an obvious thing to do, but as at the moment I write this post I am not that experienced with OpenCV, I did take some time to figure out the solution, as many of the examples available out there use Windows or Linux, with different setups.

My working compile command is this one:

gcc -o detect detect.c -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_objdetect

It does make sense linking against objdetect if you think for a while, but as the program compiled fine without it and the error message was cryptic, the path to the solution didn’t make so much sanse, though.