artodeto's blog about coding, politics and the world

Since i have to deal with a lot of configuration formats in php- or webapplications, i decided to write a conversion tool.

Today i'm happy to announce the version 1.0.0.

It is a console application (symfony/console based) and provides one command called convert. convert expects two arguemts, source and destination. The command has an option --force if destination already exists.
What do you have to provide as source or destination? For the source, you have to provide a relative path to the source configuration file. For the destination, you have to provide a relative path with file name that should contain the converted source configuration file content.

Since the conversation depends on a component, it is restricted to this capabilities. Good to know, i also maintain that component so if this cli application needs additional features, the component should be extended quite soon ;-).

The command line configuration format converter is available on github.com or via packagist.org.

Since the formater component is using the php array as exchange format between the available formats, you should know that the configuration files are underlying some restrictions. I'm working hard to create a wiki for the convertion component.

Enjoy it and feel free to report bugs or fork it.

After getting more and more into the great world of composer based systems, i tried to reuse code from my current and ongoing projects and cast them into separat components.

Right now, the follwing are available: - Symfony Console IO - Influenced by a teammate and by composer, i wanted to have IO object in my console applications. - Symfony Console Command - Since the IO component makes more sense in the command and since, again influenced by teammate and composer, i wanted to have a command code that stops me from repeating myself in different console applications. - Symfony Console Application - The last package from my lab, like the command, this package is a application with my flavor. - Component Configuration Converter - I was getting sick of converting configuration formats from one to another, so i create a component for that. Currently the following formats are available: - YAML - XML - JSON - PHPArray

All Packages are availalbe on github.com and i would be glad if you are going to test them in the real world and add some defects.

The following script provides a solution to compare two big xml files. I tried to compare a lot of xml files with a size of greater 500 megabytes with different tools. Each tool was eating up my memory and swap and finally crashed. All i want to have is "show me what is in file one and not in file two and vice versa". I've reached this goal by using a property my xml files have. Each file as nodes. Each node has a unique identifier inside. I cutting out the unique identifier tag and putting this tag, line by line, into a file. After that, i'm sorting this unique identifiers. Finally i am using diff. To create a more useful output, i'm separating the "what is only in file one" into a own file (and the same for file two).

Happy using and if you find errors, i'm ready to fix them :-).

#!/bin/bash
####
# script to compare two xml files by (unique) tag
####
# @author stev leibelt
# @since 2013-03-13
####

if [[ $# -eq 3 ]]; then
  XML_FILE_ONE="$1"
  XML_FILE_TWO="$2"
  XML_TAG="$3"

  if [[ -f "$XML_FILE_ONE"
        && -f "$XML_FILE_TWO"
        && ! -z "$XML_TAG" ]]; then
    #retrieving xml_tags per file
    #reduce xmls by lines containing the tag
    sed -n -e 's/.*<'$XML_TAG'>\(.*\)<\/'$XML_TAG'>.*/\1/p' $XML_FILE_ONE > $XML_FILE_ONE'.sed'
    sed -n -e 's/.*<'$XML_TAG'>\(.*\)<\/'$XML_TAG'>.*/\1/p' $XML_FILE_TWO > $XML_FILE_TWO'.sed'

    #sort and uniq the sed'ed files
    sort $XML_FILE_ONE'.sed' | uniq > $XML_FILE_ONE'.sort'
    sort $XML_FILE_TWO'.sed' | uniq > $XML_FILE_TWO'.sort'

    #output the differences
    diff $XML_FILE_ONE'.sort' $XML_FILE_TWO'.sort' > 'xml_diff_by_tag.diff'
    #diff --side-by-side $XML_FILE_ONE'.sort' $XML_FILE_TWO'.sort' > 'xml_diff_by_tag.diff'
    #comm -3 $XML_FILE_ONE'.sort' $XML_FILE_TWO'.sort' > 'xml_diff_by_tag.comm'

    #show only differences per file
    sed -n -e 's/^<\ \(.*\)/\1/p' 'xml_diff_by_tag.diff' > $XML_FILE_ONE'.diff.uniq'
    sed -n -e 's/^>\ \(.*\)/\1/p' 'xml_diff_by_tag.diff' > $XML_FILE_TWO'.diff.uniq'

    #sed -n -e 's/^<\(.*\)/<\1/p' 'xml_diff_by_tag.comm' > $XML_FILE_ONE'.comm.uniq'
    #sed -n -e 's/\t<\(.*\)/<\1/p' 'xml_diff_by_tag.comm' > $XML_FILE_TWO'.comm.uniq'

    #removing unused files
    rm -fr $XML_FILE_ONE'.sed' $XML_FILE_TWO'.sed' $XML_FILE_ONE'.sort' $XML_FILE_TWO'.sort'
  else
    echo 'Invalid arguments provided'
    echo 'try '$0' $xmlFileOne $xmlFileTwo $comparingTag'
  fi
else
  echo 'Invalid number of arguments provided'
  echo 'try '$0' $xmlFileOne $xmlFileTwo $comparingTag'
fi

Available on github.com.