|
Even though I've switched to using git for my day-to-day mozilla work, there are times when I have to apply patches to my mercurial tree using mq. Sometimes applying the patches fails because something changed, and it's not easy to figure out what changed, specially if the patch is large. To solve this problem, I wrote a little bash/perl script, inspect-rejects.
To use it, stick it on your $PATH, and invoke it on the source file for which rejects were generated. For example, if rejects were generated for layout/base/nsRefreshDriver.cpp, invoke it on that file:
$ hg qpush --move try-avi
applying try-avi
patching file layout/base/nsRefreshDriver.cpp
Hunk #1 FAILED at 936
1 out of 1 hunks FAILED -- saving rejects to file layout/base/nsRefreshDriver.cpp.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
errors during apply, please fix and refresh try-avi
$ inspect-rejects layout/base/nsRefreshDriver.cpp
What this does is print out the "original" version of the code (context lines and "-" lines) from the reject file into a temporary file, and then opens vimdiff with that temp file and the source file as it is in your tree. This allows you to compare what the patch thinks your file should look like and what your file actually looks like, and allows you to quickly identify differences. You can even edit your in-tree file directly in the vimdiff editor to match what the patch expects it to look like, and then run patch on the rejects file to apply it!
$ inspect-rejects layout/base/nsRefreshDriver.cpp
2 files to edit
# edit layout/base/nsRefreshDriver.cpp to match the expected context
$ pushd layout/base && patch < nsRefreshDriver.cpp.rej && popd
~/zspace/mozilla/layout/base ~/zspace/mozilla
patching file nsRefreshDriver.cpp
~/zspace/mozilla
If you do this, you'll probably want to undo your changes afterwards - I leave automating this part as an exercise to the reader.
[ 1 Comment... ]
The main thing that I'm working on these days is "Project 256meg" (aka bug 792131). The goal of the project, as the name implies, is to get Fennec working well on phones with only 256 MiB of memory. There are a large number of older ARMv6 Android devices out there which have less than the 512 MiB of memory that we currently require for Fennec. Having Fennec run well on these devices would allow all of those people to use it as well.
One of the changes I landed recently was to "zombify" tabs under certain conditions. Zombifying a tab unloads the content loaded in the tab, thereby freeing up most of the memory associated with that tab. The tab itself remains, and remembers the URL it was on, so that when the user switches to it, it will resurrect itself by re-loading the content. The behaviour is quite similar to what Android itself does with activities - it will close activities in the background silently if needed, and open them again if the user switches back to them.
In bug 784040 we added code that zombifies all of the background tabs when we get a low-memory event from the system. In bug 792143 and bug 803575, we added some additional conditions to zombify individual tabs to prevent excessive memory usage before it happens. Specifically, when a new tab is opened, the least-recently-used background tab is zombified, if it hasn't been used in some amount of time (controlled by the browser.tabs.expireTime pref; defaults to 1 hour; -1 to disable). This behaviour is enabled by default on low-memory devices and is turned on dynamically on high-memory devices after we receive the first low-memory event on those devices.
I'm planning to also add some telemetry to see how often these events get triggered in practice, so that we can tune the behaviour as we go. However if you feel like your tabs are getting turned into zombies too often for your liking, let me know (either file a bug or comment here) and we can investigate.
[ 10 Comments... ]
With the new "native Fennec" (aka Firefox for Android, v14 or higher), the UI is built using standard Android widgets and components. While this makes the browser much faster to start up and better integrated with Android, it also has a disadvantage: add-on authors are much more limited in what they can do with the Firefox UI. We currently have a few APIs, exposed to JS via the NativeWindow object, that allow add-on authors to interact with the UI. However, this is a far cry from what can be done with desktop Firefox or the older "XUL Fennec".
Recently, I put together a proof-of-concept that allows add-ons to bundle Java code into their Firefox for Android add-on and run it on demand (see bug 794479). In theory, this will allow add-on authors to provide much deeper integration with the UI. However, as with all add-on APIs, we need to be very careful with what we expose and how we allow authors to use it, so that we balance the functionality provided with how much we are constrained from changing things in the future.
I am interested in hearing from anybody, but add-on authors in particular, to see what kind of use cases they would like to see supported, so that we can see if (1) exposing such an API is worthwhile at all, and (2) what kind of functionality we should provide if we do expose the API. Please comment on this post or in bug 799631 with any thoughts you have on the topic. Thanks!
[ 9 Comments... ]
Following on from my last post, I found three more papers in ICSE 2012 that I thought were pretty relevant to stuff we're doing at Mozilla. (Unfortunately not all of the papers are freely available online but I've tried to link to the closest relevant thing that is freely available).
ReBucket: A Method for Clustering Duplicate Crash Reports Based on Call Stack Similarity is pretty much what it sounds like. It's about a way to group crash reports based on their call stacks; their method was tested on a set of 5 Microsoft products' crash report data and found to work better than the other methods they had. Sounds like something that might be applicable to our own crash stats system.
SYMake is a tool to symbolically evaluate Makefiles and find errors in/perform refactoring on them. What I found particularly noteworthy is that it can handle things like renaming variables even if the name is built from multiple substrings, which tends to happen a lot in real-world Makefiles. Unfortunately the tool only runs on Windows so it was too much work for me to try out, but it might come in handy in analyzing our Makefiles as we re-architect our build system.
Finally, What make long term contributors: willingness and opportunity in OSS community was an interesting analysis of contributors to Gnome and Mozilla. In particular, the authors correlated various factors like the type of first contribution and size of peer group with whether or not the contributor became a long-term contributor. However it's not clear to me which of the factors discussed are both (1) under our control and (2) can cause contributors to stick around (since correlation does not necessarily imply causation).
[ 2 Comments... ]
I was going through the conference proceedings for ICSE 2012 to see if there was anything particularly interesting. One of the short papers I found was called "An Integrated Bug Processing Framework" and described a tool to automatically bisect code and find regressions based on bug reports. Although it was a two-page paper and very short on implementation details, it seemed like a good idea to incorporate at Mozilla.
Tracking down changesets that introduced regressions is harder to do on the Mozilla codebase than at most other places I've worked at, because of the size and complexity of the code. Finding the regressing changeset almost always involves bisecting nightlies or code, as opposed to doing a quick code inspection and hg blame. I recently had to do this for a Fennec bug, and it was rather tedious. (I actually ended up updating mozregression to work with Fennec, and submitted a pull request which is still pending, but even so, testing each bisection build for the error was annoying).
What would be awesome is if we modified our approach to fixing regressions slightly, and made it more test-driven. Instead of bisecting the regression, patching the code, and writing a test, we could write the test first, then use that in an automated bisection script to find the regressing changeset, and then patch the code. The test would also (by definition) be a test for the patch, and could be submitted to the repository along with the fix. This would make it much easier to track down the regressing changeset, and also ensure that patches have tests to go with them.
[ 2 Comments... ]
I recently got around to running Fennec in valgrind on the Android emulator, and figured it might be useful to post the instructions so that others can do the same. For the most part this is a blatant rehash of sewardj's 2011 blog post about valgrind on android, but with some updates and tweaks. In particular, I did not bother to build a custom ROM or kernel to get this going (although doing that will probably provide better data).
First, make sure that you can run Fennec in the Android emulator. For this, you will need to download the latest version of the Android SDK (I'm using version 16) and create an AVD in the SDK manager. I created an AVD targeting "Android 4.1 - API Level 16" as this seemed to be the most recent non-proprietary-flavoured image. With this target, ARM emulation is the only one available, so I went with that. Note that using a target of level 15 gives you the choice of ARM or x86 emulation, but since Fennec on x86 seems to be still in-progress (see bug 723713) I decided not to try that approach.
Make sure you give the AVD plenty of space on the SD card and RAM (I used 1 GiB for both, but you may want to go even higher) and ensure that you have GPU emulation turned on in the hardware properties. Boot the emulator, install the APK (the emulator should show up under adb devices, and be usable like a normal device) and make sure Fennec runs.
Once you have that set up, you need to build Valgrind. Unfortunately the release tarballs available on the Valgrind website seem to be missing some files (bug 306310), so I had to check out the SVN repository and work with that directly. Grab the code using svn co svn://svn.valgrind.org/valgrind/trunk valgrind and follow the build instructions in the README.android file to build Valgrind. Note in particular the caveats with Android NDK versions; I used the latest Android NDK available for my platform (r8).
Once you have built Valgrind into the Inst folder, start the emulator AVD image you created previously, and push the Inst folder using adb push Inst / (this is also described in the README.android and README.android_emulator files).
The final piece of the puzzle needed is the Zygote/exec hook that sewardj described in his blog post. Create a start_valgrind_fennec file that looks like this, updating the package name and valgrind parameters with your specific package name and requirements.
#!/system/bin/sh
export TMPDIR=/data/data/org.mozilla.fennec_kats
exec /data/local/Inst/bin/valgrind --log-file=/sdcard/valgrind.log --error-limit=no $*
Put it in /data/local and make sure it is executable via chmod 777. Then, tell the Android system to use it as a wrapper by running the following command on the adb shell (again replacing the username with your own):
setprop wrap.org.mozilla.fennec_kats "logwrapper /data/local/start_valgrind_fennec"
Now that all the pieces are in place, you can start Fennec using am start -n org.mozilla.fennec_kats/.App and it will be running under Valgrind. If all goes well, you should see a line in adb logcat from dalvikvm executing logwrapper, and Valgrind output should start appearing in /mnt/sdcard/valgrind.log. Also, Fennec will take a really long time to start up and do anything (this is pretty much expected).
Update: When I tried running it just now as I was writing this, Valgrind crashed because it couldn't decode some instructions. I have filed bug 306297 to track this.
Update 2 (Sep 17, 2012): The above-mentioned bug has been fixed in the latest Valgrind trunk and should be included in Valgrind 3.8.1.
[ 0 Comments... ]
Ehsan was asking me today about my git workflow at Mozilla; specifically how I push a git branch to try. I figured I'd post it here since other people may find it useful.
My git workflow involves (1) an alias in my ~/.gitconfig to generate a hg-compatible patch from a git commit, and (2) a script that runs that alias on all of the patches in the branch. This is the alias I have in my ~/.gitconfig file:
[alias]
hgp = "show --binary --format=\"# HG changeset patch%n# User %an <%ae>%n%B\" -U8"
Running git hgp HEAD, for example, will output to stdout a patch that can be imported by mq.
The script to export an entire branch is at github/staktrace/moz-scripts/hgp. Simply drop it on your $PATH and update the DESTDIR variable to some folder you would like the generated patches to end up in. (It is currently hard-coded to $HOME/zspace/patches/.)
For simple usage, just run it without any arguments. All of the commits from master..HEAD will get exported in mq-compatible format to the $DESTDIR, named n-branch-sha, where n is a counter that orders the patches sequentially, branch is the name of the git branch you're on, and sha is the shortened SHA of the commit. For example:
kats@kgupta-air mozilla-git$ git checkout layerview
Switched to branch 'layerview'
kats@kgupta-air mozilla-git$ hgp
kats@kgupta-air mozilla-git$ ls -l ~/zspace/patches/
total 1440
-rw-r--r-- 1 kats staff 6115 27 Aug 20:59 1-layerview-18ea4c9
-rw-r--r-- 1 kats staff 5805 27 Aug 20:59 2-layerview-944d80a
-rw-r--r-- 1 kats staff 5068 27 Aug 20:59 3-layerview-2fcfba1
-rw-r--r-- 1 kats staff 3199 27 Aug 20:59 4-layerview-6339849
drwxr-xr-x 253 kats staff 8602 27 Aug 09:41 old
The hgp script also takes optional arguments - the first argument is taken as the number of commits to export. So running hgp 3 will export the top three patches on your current branch. The second argument is a branch name, so you can export commits from a branch other than the one you currently have checked out.
One final thing that the script does before it exports the patches, is it moves any existing patches in $DESTDIR from the current branch into a $DESTDIR/old folder. This is because when you make changes to your git commits, the SHA values will change and if the old patches aren't removed then you could end up with two patches with the name 1-layerview-xxxxxxx, and it wouldn't be obvious which one is newer.
Anyway, using the above script I can quickly export a branch into a folder, and then it's a snap to re-import it into a mercurial queue and push it to try (or any other hg tree):
hg qimport ~/zspace/patches/*layerview*
hg qgoto try # I have a "try" patch sitting at the bottom of my default queue
hg qref -e # update any try-push options that need updating
hg push -f try
[ 1 Comment... ]
Something that came up during the "Community"-focused discussion at the Boston mobile work week last week was the availability of "mentored bugs" to attract and retain new contributors. I realized after the discussion that traditionally bugs we have at Mozilla involve writing patches in C++, JS, or Python. In Firefox for Android, however, we now have a non-trivial amount of code in Java as well. As a result, there is one less roadblock for new contributors who are comfortable with Java and want to help us out.
In order to take advantage of this, I went through my list of "I'll get to them sometime" bugs and tagged a few of them as mentored bugs, and I'd encourage everybody else on the mobile team do the same. To tag a Java bug as a mentored bug, just add the following whiteboard tags to the bug: "[good first bug][lang=java][mentor=XXX]" (replacing XXX with your handle or email address). You can also see the list of current Java-language mentored bugs here:
Good first Java-language bugs
Unfortunately our codebase isn't very IDE-friendly at the moment, so people trying to use Eclipse or other Java IDEs to contribute may have a harder time, but of course, we welcome patches to improve that situation as well! :) Finally, I would just like to link to Josh Matthew's blog post on Making bugs more attractive for other people to fix for more information on what is a good mentored bug and why we should have mentored bugs.
[ 1 Comment... ]
I spent the last week in Boston at the mobile work week. It was a lot of fun, and we made a lot of progress in terms of both planning for upcoming features and just hacking away at code. One of the things I'm working on (along with the rest of the Readability 2.0 team) is cooking up some prototypes for the Readability 2.0 effort. This effort is basically focused on trying to improve the readability of text when browsing on Firefox for Android. In our current release (Firefox 14) we have an implementation of font inflation by jwir3 and dbaron. However we are exploring ways to improve this for future releases.
The TapTapWrap addon that I blogged about before was my first attempt at trying something that might help. The most recent prototype that I have takes a different approach, and overlays out a XUL iframe with the relevant content when the user double-taps on a page. The current very rough prototype ends up looking something like this:
XUL iframe overlays
Since the content nodes are copied from the original page into the XUL iframe, we are able to sidestep concerns about modifying the original page DOM or running into issues with constrained heights as we do with font inflation or other approaches. We are also free to style the content to make it more readable, similar to what we are doing with Reader Mode (in fact, you could think of this prototype as a mini-reader-mode).
Ideally, the overlays would be scrollable, and you would also be able to click on links in them. Right now, however, we are having some issues with touch event dispatching that prevents the prototype from having that behaviour, and I'm working on tracking that down.
This prototype is just one of a few different approaches that we're exploring for Readability 2.0; another that jwir3 is working on is similar to what Opera does, where we set a maximum line width on the relevant block of text, so that it wraps to the visible window (this is commonly known as "reflow-on-zoom") and is also very promising.
[ 2 Comments... ]
I wanted to try writing an add-on for the new Firefox for Android. So I did. I saw a lot of comments in the Google Play store complaining about lack of text reflow and readability in general, so I decided to implement something that might help with that. It ended up a little different from what I had initially imagined, but it does help when you try to zoom in to a block of text by double-tapping, and it's still too small to read.
Basically the add-on grabs the text you double-tapped on, and bumps up the font size on it so that it is at least 0.5 inches tall at the new zoom level, which should make it readable. If you double-tap on it again to zoom out it gets restored to how it was originally.
You can see the source on Github: staktrace/taptapwrap, or grab the XPI. It works on all currently-available versions of the new native UI Fennec (14, 15, 16). Feedback (and pull requests) are very welcome.
If you would like to try writing an add-on, I would suggest starting at this page: Addons developer guide. It took me a while to find that page among all of the other addon-related wiki pages (and in fact I started my addon my grabbing Matt Brubeck's Phony addon and replacing chunks of it with my browser.js monkey-patching).
[ 6 Comments... ]
|