RaspberryPi: mod_rewrite & mod_userdir: Always 404 (Raspbian)

Maybe you stumbled into a similar error, I've searched my self to death but now found what's going on.

TL;DR

Setup

Let's assume you have an index.php in /home/pi/public_html/foo/ and RewriteRule ^(.*)$ index.php/$1 [L] in .htaccess (essentially). The content of the index.php is not relevant.

Now, if you visit http://raspberrypi/~pi/foo/rewrite/this/plz you always get:
404 /home/pi/public_html/foo/index.php/rewrite/this/plz not found.
Note that the 404 is about http://raspberrypi/home/pi/... not /home/pi... on your filesystem!

This is because of RewriteBase, if you set that to /~pi/foo it works, see serverfault.com: Apache2 mod_rewrite + userdir. Solution found!

Background

You may now interrupt and say: But it worked on my Debian Notebook? Yup. Same here.

My first thoughts where about a missing configuration, but a diff of the RaspberryPi and Debian config showed nothing (at least nothing regarding mod_rewrite or mod_userdir). I then searched the internet for HOURS. Nothing.

Then I thought that the version 2.4.10 vs. 2.4.23 could make a difference, maybe the packaging, the patches. But, no, not packaging or patches. I downloaded the sources of .10 and .23 and diffed that. Too much for me.
Then the most glorious idea of all came to my mind: CHANGES. Not caps because angry, but because there are CHANGES_x.y.z files. And: Win! There, in CHANGES_2.4.16 it is:

  *) mod_rewrite: Improve relative substitutions in per-directory/htaccess
     context for directories found by mod_userdir and mod_alias.  These no
     longer require RewriteBase to be specified. [Eric Covener]

Thank you, Eric, absolutely non-ironic, that's really useful :)

Also (highlighted by me):

The RewriteBase directive specifies the URL prefix to be used for per-directory (htaccess) RewriteRule directives that substitute a relative path.

This directive is required when you use a relative path in a substitution in per-directory (htaccess) context unless any of the following conditions are true:

• The original request, and the substitution, are underneath the DocumentRoot (as opposed to reachable by other means, such as Alias).
• The filesystem path to the directory containing the RewriteRule, suffixed by the relative substitution is also valid as a URL path on the server (this is rare).
• In Apache HTTP Server 2.4.16 and later, this directive may be omitted when the request is mapped via Alias or mod_userdir.

-- mod_rewrite - Apache

RTFM!

TL;DR

RewriteBase /~pi/foo, for the why read the article :P