Encoding videos for Samsung mobile phones
Posted on: July 6th, 2010I have tried around a bit with encoding videos to play them on my Samsung B2100 mobile phone. I could not find any information about which format it supported, no matter what video I tried to play, it said “Unsupported content”.
Then I tried to record a video on the mobile and encode a film with ffmpeg to match the container format, video and audio codec and bitrate and same resolution and framerate. The device still complained about “Unsupported content”.
So I looked for Samsung PC Studio (an official application by them to encode videos for Samsung mobile phones) and found it after hours of search in some corner of their website. It takes a huge amount of hard disk space, requires Windows and crashes regularly. But at least it managed to convert some WMV and AVI films (it couldn’t read MKV and MOV though). The resulting video was way too dark, but at least it was “supported content”.
“Supported content” has exactly these properties:
- Filename
- Not too long, ending in
.mp4 - Container format
- MP4
- Video format
- mpeg4, 176×144, 95 kb/s, 15 fps
- Audio format
- aac (bitrate is variable, I use 64 kb/s, 128 kb/s also work)
The command to encode a video with ffmpeg is: ffmpeg -i <Input file> -s "176x144" -r 15 -ab 64k -acodec aac -b 95k -vcodec mpeg4 <Output file>
This actually resizes your video to 4:3, which might not be desirable. I have written a small shell script to perform the conversion and adding black paddings to the sides to keep the correct aspect ratio. It needs bash, bc and ffmpeg. Usage:
video2samsung <Input file> <Output directory>– The encoded file will be placed inside he output directory, with.mp4as extension. If no output directory is specified, the directory of the input file is used.video2samsung <Input file> <Output file>– Specify the filename for the encoded video.
Removing useless newlines from JSP output
Posted on: April 27th, 2010What has always bothered me the most about JSP was the hundreds of useless newlines that the output produced. Take the following example code:
<html>
<body>
<ul>
<%
for(int i=0; i<3; i++) {
%>
<li><%=i%></li>
<%
}
%>
</ul>
</body>
</html>
Now look at the output:
<html>
<body>
<ul>
<li>0</li>
<li>1</li>
<li>2</li>
</ul>
</body>
</html>
The additional newlines are inserted because the line-breaks after the %> are preserved. Not only does this look stupid, it can also create invalid XML if the newlines occur before the <?xml declaration.
One solution would be to omit the newlines after the %>, but obviously this would make correct indentation of the HTML code inside your JSP file impossible as some lines would be preceded by two additional characters.
JSP 2.1 has introduced a trimDirectiveWhitespace parameter [source], which is included in the @page directive: <%@page trimDirectiveWhitespaces="true"%>. Look what the HTML output looks like now:
<html>
<body>
<ul>
<li>0</li>
<li>1</li>
<li>2</li>
</ul>
</body>
</html>
The trimDirectiveWhitespaces parameter not only trims the newlines after the %>, but it trims all white spaces, including the indentation of the next line! I cannot imagine how anyone could have such a stupid idea.
My solution is to replace the newlines automatically in the JSP files before deploying them to the web container. This way, both the JSP file I write and the HTML code that is created look proper. To perform this task, there exists a Maven plugin called maven-replacer-plugin. This is the configuration I use in my pom.xml file:
<pluginRepositories>
<pluginRepository>
<id>maven-replacer-plugin reposoitory</id>
<url>http://maven-replacer-plugin.googlecode.com/svn/release-repo</url>
</pluginRepository>
</pluginRepositories>
<build>
<plugins>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>maven-replacer-plugin</artifactId>
<version>1.3.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>target/${project.build.finalName}/**/*.jsp</include>
</includes>
<basedir>${basedir}</basedir>
<replacements>
<replacement>
<token>(--%>)(\n)</token>
<value>$2$1</value>
</replacement>
<replacement>
<token>(%>)(\n)</token>
<value>$2$1</value>
</replacement>
</replacements>
<regexFlags>
<regexFlag>MULTILINE</regexFlag>
</regexFlags>
</configuration>
</plugin>
</plugins>
</build>
This moves the newline behind the %> before it. JSP comments (ending with --%>) are handled separately to not break them. By only moving the newline and not removing it, the line numbers in error messages are still correct. Look at the output now:
<html>
<body>
<ul>
<li>0</li>
<li>1</li>
<li>2</li>
</ul>
</body>
</html>
Unfortunately, I have not yet found a way to use this together with maven-jspc-plugin, which accesses the JSP files from the source folder directly, so the replacement does not have any effect when you pre-compile your JSP files.
Update: Feel free to copy from my adventurous way of compiling the whitespace-fixed JSP files.
Müll am Boden…
Posted on: November 21st, 2009Wenn ich groß bin, erfinde ich mal einen Bürostuhl mit gaanz großen Rollen. Einen, der nicht an jedem Kabel und an jeder Socke, die am Boden liegt, so sehr hängen bleibt, dass man fast umkippt.
Gute Nacht
Posted on: November 20th, 2009Formel, wann ein ausgeschlafener Mensch müde wird.
- Wenn die Einschlaf-Uhrzeit gestern später war als vorgestern oder der von vorgestern gleicht, werde ich zur Einschlaf-Uhrzeit von vorgestern müde.
- Wenn die Einschlaf-Uhrzeit gestern früher war als vorgestern, werde ich zur Einschlaf-Uhrzeit von gestern müde.
Oder anders ausgedrückt: Wenn ich an einem Tag außerhalb meines Schlafrhythmus zu Bette (oder anderem Schlafplatze) gehe, verändert sich am Rhythmus nichts. Wenn ich aber an zwei Tagen hintereinander aus dem Rhythmus falle, passt er sich mit einem Tag Verzögerung an. Regel 2 stimmt nur, weil man nicht früher als zur gewöhnlichen Zeit einschlafen kann, sich die Zu-Bett-Geh-Zeit aber wohl mit einem Tag Verzögerung auf die Einschlafzeit auswirken. Wer nicht ausgeschlafen ist, kann allerdings auch schon zu einer verfrühten Zu-Bett-Geh-Zeit einschlafen, diese Ausnahme ist bei Anwendung der obigen Regel zu beachten.
Funktioniert immer. Bei mir zumindest.
Making backups with Git
Posted on: November 19th, 2009I am the maintainer of a website where I have very restricted access to the files on the webspace. The only way to access them is using sftp or scp, and the only commands I may execute when connecting to the server using ssh are some basic ones like ls or echo and rsync. So the only proper way to get a backup of the files is to copy them using rsync. I’ve been looking for a way to keep these regular backups in a revision control system for a long time, but with SVN, it just looked too complicated to maintain automatically, as you have to look which directories and files have been added or removed and then svn add or svn rm them.
So I was using the rsync hardlink mechanism until now. When creating the daily backup copy, I passed the directory of the previous backup to rsync using the --link-dest parameter. rsnyc then did not create a new copy of files that existed equally in the old backup, but instead hard linked to them so they did not require any disk space.
With Git, it becomes very easy to manage the backups. I just update my Git working space to the current state of the web space and then run git add .. This way, all newly created files will be added automatically. As Git doesn’t care about directories, there is no trouble with them. When I then run git commit -a, files that have been removed from the directory will be automatically removed from Git! I don’t have to look for them separately, as I would have to in SVN.
With my old rsync system, the problem was that some large files slightly changed very often, such as an SQLite database where an entry was added. As the file changed, a hardlink could not be created and the whole new file had to be saved, which took lots of disk space over time. I don’t know if Git’s mechanism works better here by saving only the diffs of these files.
With the old rsync system it was very easy to remove old backups, I could just remove the backup directory, as hardlinks were used instead of softlinks, it was no problem when the old files were removed. I don’t know what I’ll do when my Git repository gets too big, maybe there is a way to remove old commits. Suggestions are welcome.
Update: The files on the webspace totally take about 1,1 GiB. The Git repository (in .git) with one commit takes 965 MiB. The next day’s backup takes additional 3,7 MiB on the file system with the rsync method, in Git the new commit takes about 2 MiB. Looks good so far.
Update: I backuped another directory that contains lots of text files and some sqlite databases. The directory uses 50 MiB of disk space, the Git repository with one commit only 9,9 MiB. The second backup takes additional 29 MiB with the rsync method, in the Git repository 7,1 MiB. Looks great so far.
Committing only parts of the changes in a file with Git
Posted on: November 19th, 2009Just found this article that shows Git’s possibility to only commit parts of the changes you did in a file. Just great.
Git submodules
Posted on: November 18th, 2009Next example why the current Git submodule support is completely shit. I have various PHP applications that use the same Java backend, calling it using exec(). Now each of these PHP applications has its own Git repository, the Java backend has one, too. The PHP repositories include the Java repository as submodules, so, if someone clones one of these PHP repositories, they have to run git submodule update --init after git clone.
If I make an update now in my Java library and commit that to the public repository, the new version won’t be used automatically in the PHP applications. Instead, I have to run these commands in all PHP applications:
cd java git pull cd .. git commit -a git push
After updating the working space of the Java submodule (using git pull), it appears like a modified file in the PHP application repository, so I have to commit the change.
Users of the PHP applications now cannot just run git pull to get the new version of the application (including the new version of the Java submodule), instead they have to run an additional git submodule update after that so that the working space in the submodule gets updated, too. So you have to tell your users that they can’t just git pull changes, but instead they have to run an additional command every time.
Now things get even funnier: The Java library requires an external library to work, so it includes a submodule itself. The thing is, when people download a PHP application and load the Java submodule using git submodule update --init, the submodules of the Java submodule won’t be initialised automatically. So users have to run the following commands to get a working copy of my PHP application after git clone:
git submodule update --init cd java git submodule update --init
Now imagine that the external library used by my Java library introduces a new feature that I begin to use in my Java library. I have to update the submodule of the external library in my Java library and commit that change. Then I have to update the Java library submodules in all my PHP applications and commit these. Imagine what a user of my PHP application has to run every time he wants to update his working space to a working new version:
git pull git submodule update cd java git submodule update
My projects are rather small, imagine what you’d have to do to update a working copy of a huge project…
When I use a web application on my server (such as an webmail client or phpmyadmin), I usually check out the stable branch of their SVN repository and run svn up every now and then to get new updates. I don’t need to know anything about the repository structure of the projects to do that. With Git, I would have to know in which directories I would have to update the submodules manually, or, alternatively, there could be a shell script to update the working copy, which I would have to remember to call instead of git pull. This makes things unbelievably complicated. I hope Git will at least introduce a feature that automatically fetches the submodules on git clone or git pull.
Update: Man pages on the internet list a --recursive option for both git submodule as well as git clone that does exactly this. On none of my computers, these are supported by the installed Git version yet, so it must be a very new feature. I don’t know though if the option is available for git pull or git checkout as well. I hope that it will become or already be the default behaviour of Git. Yet I am missing an option to always include the newest commit of a repository as a submodule instead of a specific one.
Update: Oviously, the --recursive option was added in Git 1.6.5, which is still marked unstable in Gentoo Portage.
SVN pain
Posted on: November 18th, 2009Ugliest SVN issue: When committing, the revision number of the working copy is not increased when it would only have to be increased by 1. I tend to never run svn update on my working copies of projects that I develop all alone. My working copy is always up to date, but the revision number is not…
Recently I’ve been removing SVN directories quite often, but everytime I want to commit the change, SVN tells me that the directories are out of date. And updating a directory that has been removed using svn rm does really strange things…
Git disadvantages
Posted on: November 16th, 2009I’ve been considering merging from Subversion to Git lately, and have finally managed to understand how Git works. A good introduction is “Understanding Git Conceptually”. Not a good introduction is the Wikipedia article, as it mainly explains what Git is trying to be, and not what it actually is.
The biggest misunderstanding is that Git is called a “distributed” or “decentralised” revision control, as opposed to a “centralised revision control”. In fact, aside from the fact that you normally have a copy of the full repository in your working copy, Git isn’t that at all. When you hear about a “decentralised” revision control system, you suspect it to work a bit like p2p file sharing, commits would be exchanged directly between developers, a central server mediating only. This is not the fact with Git, you will always have a central repository that you commit to. If you try to develop without a central repository, you will end up in a mess.
The fact that Git is trying to be decentralised without being it leads to confusions that could have been avoided by designing it as a centralised system. For example, when you create a public branch in the central repository, you cannot use that branch directly but instead have to create another local branch that “tracks” the remote branch.
The main difference between Git and Subversion is often claimed to be that in Git you have the whole repository in your working copy, whereas in Subversion you only have the newest commit. This is a minor difference in my eyes, the main difference (and advantage) is that Git has native support for branches (whereas you have to emulate branch behaviour by duplicating directories in Subversion), and these branches can optionally only exist locally. It is very easy to create different branches for different minor changes, and you can develop on them even when you don’t have an internet connection.
The only disadvantage of Git compared to Subversion I have come across is a huge one, and you should really consider it before merging to Git. Git comes with very useful functionality to participate on the development of a project. However, one other very important use of Subversion is to get a copy of a project or a part of it (such as a library) and to easily keep up to date with changes without dropping your own local hacks (something like a “read-only working copy”, you never intend to commit any changes). Git is currently not made for this use at all, as you always have to download a lot more than you need. There is in fact an option to avoid downloading old commits that you don’t need (using --depth=1), but there is no way to only download a specified sub-directory. Common practice in Git is to create an own repository for every part of a project that one might want to check out on its own, and then to include these repositories using submodules (something similar to SVN externals). The problem about that is that it creates a lot of work in Git to make a change in one submodule, commit that and then load the changes into the other repositories including that. And for many projects, it is just impractical to split the code into multiple repositories. If I want to include a part of a large project (such as a part of a library) into my project, I have to include that whole project, which can take hours to download given today’s slow internet connections. There might be workarounds around this, but they certainly aren’t as simple as a single “svn up”.
So the major advantage of SVN over Git is that it is very easy and fast to get a complete and up-to-date “read-only” copy of a project by just using “svn co” or “svn up”. In Git, you have to clone the repository, then it might be incomplete because you have to additionally initialise the sub-modules. And downloading those might take hours.
As long as this disadvantage persists in Git, many projects will keep using SVN. And as long as these projects keep using SVN, it will be difficult for other projects to merge to Git, because they reference these projects using svn:externals.
I hope that these possibilities will be included in future versions of Git:
- Download a “read-only” copy of a Git repository or one of its sub-directories, with automatic initialisation of its sub-modules. (The copy should of course still be updatable using “git pull”.) Transmitting old commits and other useless bandwidth usage should be avoided.
- Something like svn:externals. Something that automatically pulls the newest commit of a repository or its sub-directory into a sub-directory of my working copy.
- SVN support for this svn:externals-like thing. It is already well possible to clone a subversion repository, so it shouldn’t be a problem to support importing them.
Gründe für den Datenschutz
Posted on: May 3rd, 2009Im Gegensatz zu sinnlos einschränkenden Gesetzen wie dem Inzest-, Drogen-, Pornografieverbot und dem Ladenschlussgesetz sind es in Bezug auf die totale Überwachung die Gegner, die hier ihren Standpunkt niemals vernünftig begründen und ihn als offensichtlich darstellen. Hier also zwei vernünftige Gründe für den Datenschutz.
Grund 1: Nicht alles, was verboten ist, ist auch schlimm. Beispiele:
- Jemand hat eine Liebesbeziehung zu seinem Adoptiv- oder gleichgeschlechtlichen Geschwister.
- Jemand raucht ab und zu aus Spaß Marihuana, ohne davon abhänig zu sein.
- Der einzige Fahrer um 3 Uhr nachts auf einem Autobahnstück, auf dem 100 km/h erlaubt sind, fährt 150.
- Ein muslimischer Mann heiratet mehrere Frauen.
- Ein minderjähriges Pärchen hat in den USA Geschlechtsverkehr.
- Jemand fährt tagsüber mit einem Fahrrad durch die Stadt, das keinen Dynamo hat.
- Ein Naturfreund übernachtet auf einer Reise fernab der Zivilisation in einem Wald.
- Ein Obdachloser übernachtet auf einer Parkbank.
All diese „Verbrechen“ werden in unserer Welt mehr oder weniger schlimm bestraft, einige davon mit Gefängnis. Die Gesetze hierzu werden von unserer Gesellschaft nur geduldet, weil jeder weiß, dass in diesen „minderschweren“ Situationen niemand sie kontrolliert. In einer Gesellschaft der totalen Überwachung wird jedes Verbrechen entdeckt und bestraft, und solange Gesetze keine Ausnahmen in minderschweren Fällen machen und der Trend weiterhin dahin geht, den Bürgern immer mehr Sachen zu verbieten, wird die Freiheit dadurch in großem Maße unnütz eingeschränkt. Weiterhin wird in unserer Gesellschaft diverses „unsittliches“ Verhalten geächtet, und wenn Daten in private Hände gelangen, wird man unter Umständen von der Gesellschaft für sein Verhalten „bestraft“. Darüberhinaus kann jegliches Gesetz (zum Beispiel ein Verbot der öffentlichen Bekenntnis zu einer politischen Richtung) nach der (illegalen) Einführung sofort durchgesetzt und damit das rechtliche Vorgehen dagegen verhindert werden.
Grund 2: Aus Risikofaktoren lässt sich Profit schlagen, diese sind jedoch keine Ursachen. Beispiele:
- Anhand des Kaffeekonsums (überwachbar zum Beispiel bei Auswertung der Einkäufe mit elektronischem Geld) eines Menschen lässt sich statistisch auf den Stress schließen, den dieser in der Arbeit hat. Da Stress Krankheiten verursacht, würden seine Krankenkassenbeiträge erhöht oder ihm im Krisenfall sogar eine Krankenversicherung verwährt, und das ebenso jemandem, der keinen Stress hat und Kaffee aus Genussgründen trinkt.
- Jemand, der häufig Besuch aus dem islamischen Ausland bekommt (nachweisbar zum Beispiel durch RFID-Chips in Reisepässen), ist statistisch gesehen wahrscheinlicher ein Mitglied einer islamistischen Organisation. Dadurch geraten auch Unschuldige (zum Beispiel solche, die viele Geschäftspartner einladen) unter Verdacht, und unter einer großen Zahl unschuldiger Verdächtiger gibt es aufgrund der Ermittlungs-Fehlerquote immer ein paar, die schuldig gesprochen werden.
Die Menschen dürften also aus statistischen Gründen bestimmte Dinge nicht mehr tun, weil sie sonst unter Verdacht geraten würden. Wer nun sagt, dass die Menschen nunmal in unserer Welt auf bestimmte Dinge verzichten müssten, um die Sicherheit sicherzustellen und Verbrechen zu bekämpfen, dem sei gesagt, dass der Sinn unserer Staaten und das Hauptziel unserer selbst immernoch nicht ist, in einer Welt ohne Verbrechen zu leben, sondern das zu tun und tun zu können, was wir wollen.