Jekyll2022-08-04T12:18:05+00:00https://raphael.medaer.me/feed.xmlRaphael Medaer’s blogHi! I'm Raphael Medaer and here is my blog. It's been a while since I wanted to share some of my ideas or experiments. This pages are (finally) the realization of this project.
Prevent Forwarded header spoofing with HTTP message signature2022-08-04T11:00:00+00:002022-08-04T11:00:00+00:00https://raphael.medaer.me/2022/08/04/prevent-forwarded-header-spoofing<p>The <code class="language-plaintext highlighter-rouge">Forwarded</code> HTTP header has been introduced in <a href="https://www.rfc-editor.org/rfc/rfc7239.html">RFC7239</a> from June 2014. It <em>“defines an HTTP extension header field that allows proxy components to disclose information lost in the proxying process, for example, the originating IP address(…)”</em>.
However there is not any mechanism to protect subsequent component against spoofing. Indeed, if your subsequent component, let’s say an API, is exposed on Internet, anyone can forge a HTTP <code class="language-plaintext highlighter-rouge">Forwarded</code> header. If your API is always behind your proxy, it has to remove/replace the original <code class="language-plaintext highlighter-rouge">Forwarded</code> header. It becomes even more complex if you have multiple proxies.</p>
<!--more-->
<p>What could you do to mitigate this security issue ? You could<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>:</p>
<ul>
<li>restrict access to subsequent components with network filtering (firewall, private network, …)</li>
<li>allow the <code class="language-plaintext highlighter-rouge">Forwarded</code> header only for a given set of whitelisted IP</li>
<li>identify and trust your proxy using mTLS, API key,…</li>
</ul>
<p>These solutions have a few drawbacks, especially if you have multiples proxies in cascade. However we could think about another solution based on an upcoming RFC (still in draft): the <a href="https://datatracker.ietf.org/doc/draft-ietf-httpbis-message-signatures/"><em>“HTTP Message Signatures” (draft-ietf-httpbis-message-signatures)</em></a>. As explained in the abstract, it <em>“supports use cases where the full HTTP message may not be known to the signer, and where the message may be transformed (e.g., by intermediaries) before reaching the verifier”</em>, which is the case of the <code class="language-plaintext highlighter-rouge">Forwarded</code> header.</p>
<p>How does it work ? The idea is that the <a href="https://www.ietf.org/archive/id/draft-ietf-httpbis-message-signatures-11.html#name-conventions-and-terminology">“verifier”</a>, basically our subsequent component (for instance an API) will check the integrity of the <code class="language-plaintext highlighter-rouge">Forwarded</code> header with a digital signature of this header stored in two different headers named <code class="language-plaintext highlighter-rouge">Signature</code> and <code class="language-plaintext highlighter-rouge">Signature-Input</code>. The <code class="language-plaintext highlighter-rouge">Signature-Input</code> contains the metadata of the message signature, in a nutshell: a signature identifier, the signed components (headers, method, uri,…) and some parameters (algorithm used to sign, key identifier, creation date and expiration date). In our case, it will only contain the component derived from the <code class="language-plaintext highlighter-rouge">Forwarded</code> header:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Signature-Input: proxy_signature=("forwarded")\
;created=1618884480;expires=1618884540;keyid="my-proxy-key-rsa-example"\
;alg="rsa-v1_5-sha256"
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">Signature</code> header contains the signature of the (well formated) components using the private key.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Signature: proxy_signature=:G1WLTL4/9PGSKEQbSAMypZNk+I2dpLJ6qvl2JISahlP31OO/QEUd8/\
HdO2O7vYLi5k3JIiAK3UPK4U+kvJZyIUidsiXlzRI+Y2se3SGo0D8dLfhG95bKr\
(...)
M9P7WaS7fMGOt8h1kSqgkZQB9YqiIo+WhHvJa7iPy8QrYFKzx9BBEY6AwfStZAs\
XXz3LobZseyxsYcLJLs8rY0wVA9NPsxKrHGA==:
</code></pre></div></div>
<p>Therefore, the signature generated by our proxy to prove the <code class="language-plaintext highlighter-rouge">Forwarded</code> header validity will be used by the subsequent component (aka our API) to check for instance the origin IP. Obviously, the proxy private key must remain a secret and our API MUST know the proxy’s public key. This mechanism could be use by multiple proxies in cascade to sign their message transformations.</p>
<p>But… I will close this post with an important disclaimer: this (probably-not-new) idea came to me while I was reading the draft written by Annabelle Backman, Justin Richer and Manu Sporny. Indeed, the example above is coming from the section <a href="https://www.ietf.org/archive/id/draft-ietf-httpbis-message-signatures-11.html#name-multiple-signatures"><em>“4.3. Multiple Signatures”</em> of the draft</a>. Furthermore, I didn’t test any implementation and there are a lot of security concerns to consider before going further with this fun but experimental feature.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>Some of the solutions described here are coming from the <a href="https://security.stackexchange.com/questions/254199/how-to-prevent-spoofing-of-x-forwarded-for-header">following Security StackExchange question: <em>“How to prevent spoofing of X-Forwarded-For header?”</em></a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>The Forwarded HTTP header has been introduced in RFC7239 from June 2014. It “defines an HTTP extension header field that allows proxy components to disclose information lost in the proxying process, for example, the originating IP address(…)”. However there is not any mechanism to protect subsequent component against spoofing. Indeed, if your subsequent component, let’s say an API, is exposed on Internet, anyone can forge a HTTP Forwarded header. If your API is always behind your proxy, it has to remove/replace the original Forwarded header. It becomes even more complex if you have multiple proxies.New music project2022-04-01T06:00:00+00:002022-04-01T06:00:00+00:00https://raphael.medaer.me/2022/04/01/new-music-project<p>I am proud to present a new music project named “Brunch”. This is a brand new single which will be released on May 1st featuring two chill and smooth piano tracks with inspirations from Erik Satie to Michel Petrucciani. <a href="https://distrokid.com/hyperfollow/raphaelmedaer/brunch">More to come soon…</a></p>I am proud to present a new music project named “Brunch”. This is a brand new single which will be released on May 1st featuring two chill and smooth piano tracks with inspirations from Erik Satie to Michel Petrucciani. More to come soon…Fun with Node experimental modules and loaders2022-01-28T13:00:00+00:002022-01-28T13:00:00+00:00https://raphael.medaer.me/2022/01/28/fun-with-node-experimental-modules-and-loaders<p>If you are a front-end or a fullstack developer, you maybe already imported CSS files from a Javascript or a Typescript file. For instance:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">styles</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">./styles.css</span><span class="dl">"</span>
</code></pre></div></div>
<p>A few months ago it would not have been possible out of the shelf. Indeed you would need a bundler (such as WebPack, Rollup,…) to “inline” the CSS file as a string in your Javascript file.</p>
<p>Nowadays it’s possible thanks to the “CSS Modules” (to not confuse with <a href="https://github.com/css-modules/css-modules">the homonym open-source project</a>).<!--more--> It’s even already <a href="https://chromestatus.com/feature/5948572598009856">implemented in Chrome</a>. For more details, please read the <a href="https://css-tricks.com/css-modules-the-native-ones/"><em>“CSS Modules (The Native Ones)”</em> article</a> from <a href="https://css-tricks.com/author/chriscoyier/">Chris Coyier</a>. However it’s not yet brought to NodeJS implementation. In the meantime, NodeJS allow us to <a href="https://nodejs.org/api/esm.html#loaders">customize the default module resolution</a> through three JS hooks: <a href="https://nodejs.org/api/esm.html#resolvespecifier-context-defaultresolve"><code class="language-plaintext highlighter-rouge">resolve</code></a>, <a href="https://nodejs.org/api/esm.html#loadurl-context-defaultload"><code class="language-plaintext highlighter-rouge">load</code></a> and <a href="https://nodejs.org/api/esm.html#globalpreload"><code class="language-plaintext highlighter-rouge">globalPreload</code></a>. These hooks are provided via the command line argument <code class="language-plaintext highlighter-rouge">--experimental-loader</code>. Pay attention that it’s an experimental feature. Furthermore the <em>“API is currently being redesigned and will still change”</em>.</p>
<p>We will use this feature to load CSS file as an ECMAScript module and therefore bypass the bundling/building phase. We will also have to use ECMAScript modules to make it work:</p>
<blockquote>
<p>To load an ES module, set “type”: “module” in the package.json or use the .mjs extension.</p>
</blockquote>
<p>Our project starts with the following files:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* index.mjs */</span>
<span class="k">import</span> <span class="nx">styles</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">./styles.css</span><span class="dl">"</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">styles</span><span class="p">);</span>
</code></pre></div></div>
<p>and</p>
<div class="language-css highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">/* styles.css */</span>
<span class="nt">html</span><span class="o">,</span> <span class="nt">body</span> <span class="p">{</span>
<span class="nl">margin</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Without custom loader, the result would be an <em>“Unknown file extension”</em>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>node index.mjs
node:internal/errors:464
ErrorCaptureStackTrace<span class="o">(</span>err<span class="o">)</span><span class="p">;</span>
^
TypeError <span class="o">[</span>ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension <span class="s2">".css"</span> <span class="k">for</span> /<span class="o">(</span>...<span class="o">)</span>/styles.css
at new NodeError <span class="o">(</span>node:internal/errors:371:5<span class="o">)</span>
<span class="o">(</span>...<span class="o">)</span>
at async ModuleWrap.<anonymous> <span class="o">(</span>node:internal/modules/esm/module_job:81:21<span class="o">)</span> <span class="o">{</span>
code: <span class="s1">'ERR_UNKNOWN_FILE_EXTENSION'</span>
<span class="o">}</span>
Node.js v17.3.0
</code></pre></div></div>
<p>Let’s write our basic custom loader:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* loader.mjs */</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">URL</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">url</span><span class="dl">"</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">readFile</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">fs/promises</span><span class="dl">"</span><span class="p">;</span>
<span class="cm">/**
* This function loads the content of files ending with ".css" to an ECMAScript Module
* so the default export is a string containing the CSS stylesheet.
*/</span>
<span class="k">export</span> <span class="k">async</span> <span class="kd">function</span> <span class="nx">load</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="nx">context</span><span class="p">,</span> <span class="nx">defaultLoad</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">url</span><span class="p">.</span><span class="nx">endsWith</span><span class="p">(</span><span class="dl">"</span><span class="s2">.css</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">content</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">readFile</span><span class="p">(</span><span class="k">new</span> <span class="nx">URL</span><span class="p">(</span><span class="nx">url</span><span class="p">));</span>
<span class="k">return</span> <span class="p">{</span>
<span class="na">format</span><span class="p">:</span> <span class="dl">"</span><span class="s2">module</span><span class="dl">"</span><span class="p">,</span>
<span class="na">source</span><span class="p">:</span> <span class="s2">`export default </span><span class="p">${</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">content</span><span class="p">.</span><span class="nx">toString</span><span class="p">())}</span><span class="s2">;`</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">defaultLoad</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="nx">context</span><span class="p">,</span> <span class="nx">defaultLoad</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Running NodeJS with our loader will now print<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> the content of <em>styles.css</em>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>node <span class="nt">--no-warnings</span> <span class="nt">--experimental-loader</span> ./loader.mjs index.mjs
html, body <span class="o">{</span>
margin: 0<span class="p">;</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This is a really basic example to understand NodeJS custom loaders. As you may have noticed, I didn’t use <a href="https://github.com/tc39/proposal-import-assertions"><em>“Import Assertions”</em></a> although it’s mandatory for <a href="https://nodejs.org/api/esm.html#json-modules">JSON modules</a> in NodeJS. For security reasons the import should actually look like:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">styles</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">./styles.css</span><span class="dl">"</span> <span class="nx">assert</span> <span class="p">{</span> <span class="nl">type</span><span class="p">:</span> <span class="dl">"</span><span class="s2">css</span><span class="dl">"</span> <span class="p">};</span>
</code></pre></div></div>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>The argument <code class="language-plaintext highlighter-rouge">--no-warnings</code> is used for ease of reading, I recommend you to keep warnings in your development and deployments. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>If you are a front-end or a fullstack developer, you maybe already imported CSS files from a Javascript or a Typescript file. For instance: import styles from "./styles.css" A few months ago it would not have been possible out of the shelf. Indeed you would need a bundler (such as WebPack, Rollup,…) to “inline” the CSS file as a string in your Javascript file. Nowadays it’s possible thanks to the “CSS Modules” (to not confuse with the homonym open-source project).Yet another successful Git branching model2021-03-22T00:00:00+00:002021-03-22T00:00:00+00:00https://raphael.medaer.me/2021/03/22/yet-another-successful-git-branching-model<p>A decade ago, <a href="https://twitter.com/nvie">Vincent Driessen</a> wrote a post named <a href="https://nvie.com/posts/a-successful-git-branching-model/"><em>“A successful Git branching model”</em></a>. In his post he presents a development model using many Git branches (<em>master</em>, <em>develop</em>, …) well known as the <em>Git flow</em>. Even if I’m not doing professional development for ten years, I got the opportunity to experiment Git flows in many development teams. As explained by his <a href="https://nvie.com/posts/a-successful-git-branching-model/"><em>“Note of reflection (March 5, 2020)”</em></a>, you should <strong>not</strong> treat this flow as a dogma or panacea. While the flow I will describe in the following lines is inspired by Vincent’s Git flow, it differs in many ways to fit the requirements of development team I’m working with right now at <a href="https://allocloud.com">ALLOcloud</a>.
<!--more--></p>
<p>The repository we will talk about has 14,300+ commits, 242.3MB of files and 8 regular contributors (on a daily basis). It contains the code and resources of a heavily used cloud platform including front-end, backend and deployment-as-code developments. It’s some-how a mono-repository. When a developer is working on a new feature, he might have to write code in Javascript or Python but also YAML for Ansible and SQL migration files.</p>
<h3 id="development-vs-deployment">Development vs Deployment</h3>
<p>A few months ago, when I joined <a href="https://allocloud.com">ALLOcloud</a> team, we were mixing development, release and deployment processes all together. It was, therefore, difficult to know what is in production, what we can/will deploy, what needs to be tested and how tricky will be our next deployment. As a consequence our platform updates were planned on monthly basis with a lot of stress and required a lot of effort (and production patches).</p>
<p>Now we split our development and deployment processes. Thanks to GitLab, we maintain our development cycle in Git and we manage our deployments with <a href="https://docs.gitlab.com/ee/ci/README.html">GitLab CI</a> and its <a href="https://docs.gitlab.com/ee/ci/environments/">Environments feature</a>. No more branches for a specific production environment, no more useless merges when we need to deploy, no more duplicated commits or cherry-picks.</p>
<h3 id="our-git-flow">Our Git flow</h3>
<h4 id="the-main-branches">The main branches</h4>
<p>The Git repository is organized with two main branches which have more-or-less the same behavior as the <cite>Git flow</cite>:</p>
<ul>
<li>the <em>master</em> branch: each commit in this branch represents a unique production release. It must be a merge commit without fast-forward.</li>
<li>the <em>develop</em><sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> branch: this branch contains atomic feature/bug-fix commits ready to be delivered in production. It means that a commit in <em>develop</em> branch is tested and ready to go in production for our next release.</li>
</ul>
<p>These two branches already differ from original Git flow because commits in <em>develop</em> branch are already production-ready. It makes the <strong><em>hot-fix</em> branches unnecessary</strong> because we can always release from <em>develop</em> branch (bringing sometimes new features while we deploy a hot-fix).</p>
<h4 id="development-of-a-new-feature-or-a-bug-fix">Development of a new feature or a bug fix</h4>
<p>Any new development (feature or bug-fix) start with a new working branch from <em>develop</em>. Branches MUST follow <a href="#branches">the branch name convention</a>. For instance:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git checkout <span class="nt">-b</span> features/example develop
</code></pre></div></div>
<p>It’s now time to write and commit your changes according to the <a href="#commits">commit conventions</a>. Usually I recommend to use <code class="language-plaintext highlighter-rouge">git add -p</code> instead of <code class="language-plaintext highlighter-rouge">git add .</code>. That avoids non-related changes and non-atomic commits. Commit as much as you need to achieve your brand new feature. Don’t forget to regulary push your branch.</p>
<p>Now your feature is ready and tested, it’s time to merge in <em>develop</em> branch. Here is how we are merging our changes to avoid conflicts, issues and crappy graph:</p>
<ol>
<li>
<p>Rebase your branch on <em>develop</em>. You may have to fix conflicts. Because they are solved during the rebase, you won’t have to do it during the merge. It will also help to keep the Git graph clean and linear.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git fetch <span class="nt">--all</span>
git rebase <span class="nt">--keep-empty</span> <span class="nt">--rebase-merges</span> origin develop
</code></pre></div> </div>
<p>If you’re working with “Merge request” feature, I recommend you to push to remote repository. Because we rebased our branch, we have to force the push: <code class="language-plaintext highlighter-rouge">git push -f</code><sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">2</a></sup>.</p>
</li>
<li>
<p>Now we are ready to execute <code class="language-plaintext highlighter-rouge">git merge</code> command. Although we have 2 solutions: a fast-forward or a non-fast-forward merge. We want to keep the whole history of our work (including atomic commits from working branches) so we will prefer non-fast-forward merges (<code class="language-plaintext highlighter-rouge">--no-ff</code>). However if your branch contains only one single commit, it’s useless to create a new merge commit. In this case we will only accept fast-forward merge (<code class="language-plaintext highlighter-rouge">--ff-only</code>).</p>
<p>You can list the number of commits in your branch with: <code class="language-plaintext highlighter-rouge">git cherry -v develop</code></p>
<p>So there are actually 2 ways to merge:</p>
<ul>
<li>
<p>If there is <strong>only one commit</strong> in your branch, merge with <strong>fast-forward only</strong>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git checkout develop
git merge <span class="nt">--ff-only</span> <your_branch>
</code></pre></div> </div>
</li>
<li>
<p>If there are <strong>multiple commits</strong> in your branch, merge with <strong>no fast-forward</strong> option:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git checkout develop
git merge <span class="nt">--no-ff</span> <span class="nt">--edit</span> <your_branch>
</code></pre></div> </div>
<p>Pay attention to the <code class="language-plaintext highlighter-rouge">--edit</code><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">3</a></sup> argument. It allows you to edit your merge commit message according to the <a href="#commits">commit conventions</a>.</p>
</li>
</ul>
</li>
</ol>
<p>Merge is done. You can now push and clean-up your branch.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git push
git branch <span class="nt">-D</span> <your_branch>
git push <span class="nt">--delete</span> origin <your_branch>
</code></pre></div></div>
<p>Compared to Git flow, we create a commit object <strong>only</strong> if we have more than 1 commit in the working branch. We also avoid many merge conflicts (usually resolved in merge commit) thanks to the rebase-before-merge.</p>
<h4 id="releases">Releases</h4>
<p>When the time has come to do a release, we merge from <em>develop</em> to <em>master</em> with a no-fast-forward merge (<code class="language-plaintext highlighter-rouge">--no-ff</code>). Thanks to <a href="#commits">commit conventions</a> and adaptative merge-fast-forward strategy, I can list the features I will release before merging:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git log <span class="nt">--oneline</span> <span class="nt">--first-parent</span> master..HEAD <span class="c"># from develop branch</span>
</code></pre></div></div>
<p>I would also recommend you to tag your releases. Currently we tag the platform releases with a date.</p>
<h3 id="conclusion">Conclusion</h3>
<p>We are now running this flow for a few months. The migration was not an easy task. Changing habits might take a while, but the result is a huge success. Even if the merge strategy is a little bit complex<sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">4</a></sup>, we do have a clean repository history with more frequent releases. There are probably some drawbacks that we are not yet aware of. However for now this flow is working fine <strong>for our needs</strong>.</p>
<hr />
<h3 id="conventions">Conventions</h3>
<h4 id="branches">Branches</h4>
<p>Working branch names are written with Kebab case and contains 2 parts: <code class="language-plaintext highlighter-rouge"><directory>/<name></code></p>
<ul>
<li><strong>directory</strong>: category of the branch (<em>features</em>, <em>bug-fixes</em>, …)</li>
<li><strong>name</strong>: a very short meaning full name</li>
</ul>
<p>For instance:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">features/order-items</code></li>
<li><code class="language-plaintext highlighter-rouge">bug-fixes/quote-original-author</code></li>
</ul>
<h4 id="commits">Commits</h4>
<p>We follow a few rules about our commits:</p>
<ol>
<li>A commit MUST be atomic</li>
<li>A commit MUST NOT contain unrelated changes</li>
<li>A commit MUST NOT break the build</li>
</ol>
<p>The commit message MUST follow the rules defined by <a href="https://twitter.com/cbeams">Chris Beams</a> in his post <a href="https://chris.beams.io/posts/git-commit/"><em>“How to Write a Git Commit Message”</em></a>:</p>
<blockquote class="citation" cite="https://chris.beams.io/posts/git-commit/#seven-rules">
<ol>
<li>Separate subject from body with a blank line</li>
<li>Limit the subject line to 50 characters</li>
<li>Capitalize the subject line</li>
<li>Do not end the subject line with a period</li>
<li>Use the imperative mood in the subject line</li>
<li>Wrap the body at 72 characters</li>
<li>Use the body to explain what and why vs. how</li>
</ol>
</blockquote>
<h3 id="faq">FAQ</h3>
<ul>
<li>
<p><em>Why not always merge with fast-forward ?</em></p>
<p>We use <code class="language-plaintext highlighter-rouge">git log --first-parent</code> on <em>develop</em> branch to list all features and bug-fixes. With fast-forward merges we would list all the commits of each branch while we only want the summary.</p>
</li>
<li>
<p><em>Do you deploy automatically your changes ?</em></p>
<p>Release commits in <em>master</em> branch are <strong>not</strong> deployed automatically. It requires a manual action to deploy on each production environment. Creating a (non-fast-forward merge) commit in this branch only means that we are ready to deploy this state in production.</p>
<p>Commits in <em>develop</em> branch trigger automatic deployment on our staging environment.</p>
</li>
<li>
<p><em>How can I list changes of a given release ?</em></p>
<p>Whereas release tag is <code class="language-plaintext highlighter-rouge">releases/2021-03-08</code>, you can execute command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git log <span class="nt">--oneline</span> <span class="nt">--first-parent</span> releases/2021-03-08~..<span class="si">$(</span>git merge-base releases/2021-03-08 develop<span class="si">)</span>
</code></pre></div> </div>
</li>
</ul>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>For legacy reason we still call this branch <em>staging</em>. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>Thanks to GitLab, <em>master</em> and <em>develop</em> are protected, <code class="language-plaintext highlighter-rouge">git push -f</code> is therefore forbidden on these branches. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>The <code class="language-plaintext highlighter-rouge">--edit</code> argument is not required to open your editor. It’s already done with <code class="language-plaintext highlighter-rouge">--no-ff</code>. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>It would be a good idea to write some tooling around Git to automate this merges. <a href="#fnref:4" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>A decade ago, Vincent Driessen wrote a post named “A successful Git branching model”. In his post he presents a development model using many Git branches (master, develop, …) well known as the Git flow. Even if I’m not doing professional development for ten years, I got the opportunity to experiment Git flows in many development teams. As explained by his “Note of reflection (March 5, 2020)”, you should not treat this flow as a dogma or panacea. While the flow I will describe in the following lines is inspired by Vincent’s Git flow, it differs in many ways to fit the requirements of development team I’m working with right now at ALLOcloud.Configure your sockets with URIs2020-08-20T08:42:18+00:002020-08-20T08:42:18+00:00https://raphael.medaer.me/2020/08/20/define-socket-w-uris<p>Everybody knows Uniform Resource Identifier (URI)<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>. As its name suggests, it’s a way to identify a resource (for instance a file or a phone number).
It is often confused with Uniform Resource Locator (URL), which is actually a form of URI. A URL is <em>a compact string representation for a resource available via the Internet</em><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>.</p>
<p>Today I want to share with you the way I define another type of resources: network sockets.
<!--more-->
Everybody knows what a network socket is, right ?</p>
<p>The socket families I often use are <code class="language-plaintext highlighter-rouge">inet</code>, <code class="language-plaintext highlighter-rouge">inet6</code> and <code class="language-plaintext highlighter-rouge">unix</code>. But there are much more types of endpoints: <code class="language-plaintext highlighter-rouge">ipx</code>, <code class="language-plaintext highlighter-rouge">bluetooth</code>,…</p>
<p>To configure the sockets my services or APIs have to bind, I’m now using URIs! For instance:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>inet://127.0.0.1:80?reuseaddr=true
</code></pre></div></div>
<p>In this example I basically create a socket IPv4 on address <code class="language-plaintext highlighter-rouge">127.0.0.1</code> and port <code class="language-plaintext highlighter-rouge">80</code> with <code class="language-plaintext highlighter-rouge">SO_REUSEADDR</code> option enabled.</p>
<p><strong>PRO:</strong> It might be specified in any configuration file because it’s basically just a string. You don’t need any complex structure to define socket type or options.</p>
<h3 id="syntax">Syntax</h3>
<p>The URI <em>scheme</em> defines which socket family I have to use:</p>
<ul>
<li><em>inet</em> for IPv4 sockets</li>
<li><em>inet6</em> for IPv6 sockets</li>
<li><em>unix</em> for Unix sockets</li>
</ul>
<p>Hopefully these schemes are not (yet) registered at IANA<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>.</p>
<p>The URI’s <em>authority</em> and <em>path</em> depend on the family used. Query arguments are used to configure options of the socket. They also depend on the family used.</p>
<h4 id="ipv4-sockets">IPv4 sockets</h4>
<p>For <em>inet</em> family the authority is mandatory. It contains:</p>
<ul>
<li>the <em>hostname</em> (optional):
<ul>
<li>either a IPv4, for instance <code class="language-plaintext highlighter-rouge">127.0.0.1</code></li>
<li>or a hostname which needs to be resolved as a IPv4 address</li>
<li>if not specified, the default value is <code class="language-plaintext highlighter-rouge">0.0.0.0</code>: it binds on all IPv4 addresses</li>
</ul>
</li>
<li>the <em>port</em> (mandatory)</li>
</ul>
<p>The <em>path</em> is not used in this case, it remains empty.</p>
<p>Here are some examples:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">inet://127.0.0.1:8080</code> binds on address <code class="language-plaintext highlighter-rouge">127.0.0.1</code> and port <code class="language-plaintext highlighter-rouge">8080</code>, easy!</li>
<li><code class="language-plaintext highlighter-rouge">inet://example.com:5432</code> resolves <code class="language-plaintext highlighter-rouge">example.com</code> to an IPv4 and binds on port <code class="language-plaintext highlighter-rouge">5432</code></li>
<li><code class="language-plaintext highlighter-rouge">inet://:80?reuseaddr=true</code> listen on port <code class="language-plaintext highlighter-rouge">80</code> and accept connections on any IPv4 of the system (<code class="language-plaintext highlighter-rouge">0.0.0.0</code>) even if there are remaining connections in <code class="language-plaintext highlighter-rouge">TIME_WAIT</code></li>
</ul>
<h4 id="ipv6-sockets">IPv6 sockets</h4>
<p>Idem than <code class="language-plaintext highlighter-rouge">inet</code> but for IPv6 and with URI scheme <code class="language-plaintext highlighter-rouge">inet6</code>. Some query argments might defer.</p>
<p>You’re probably wondering why I use <code class="language-plaintext highlighter-rouge">inet</code> <strong>and</strong> <code class="language-plaintext highlighter-rouge">inet6</code>. Indeed I could use an IPv6 address with the <code class="language-plaintext highlighter-rouge">inet</code> scheme. However it doesn’t work with domain name resolution. For instance <code class="language-plaintext highlighter-rouge">inet://localhost:80</code> will bind on IPv4 address <code class="language-plaintext highlighter-rouge">127.0.0.1</code> while <code class="language-plaintext highlighter-rouge">inet6://localhost:80</code> will bind on IPv6 address <code class="language-plaintext highlighter-rouge">::1</code>.</p>
<h4 id="unix-sockets">Unix sockets</h4>
<p><em>Authority</em> part is not used for <em>unix</em> family. However the path is used like a <code class="language-plaintext highlighter-rouge">file:/</code> URI. It defines the path of the unix socket.</p>
<p>For instance: <code class="language-plaintext highlighter-rouge">unix:///var/run/my-socket.sock</code></p>
<h3 id="specification--implementation">Specification & implementation</h3>
<p>I don’t think there is already a specification or RFC which tackle this idea.</p>
<p>I guess this trick is already used in <del>some</del> many implementations. To be honnest I didn’t look a lot. But feel free to share your godsend!</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>See <a href="https://tools.ietf.org/html/rfc3986">RFC3986 “Uniform Resource Identifier (URI): Generic Syntax”</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Quoting <a href="https://tools.ietf.org/html/rfc1738#section-1">RFC1738 “Uniform Resource Locators (URL)”, Section 1 “Introduction”</a> <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>See <a href="https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml">IANA Assignements: Uniform Resource Identifier (URI) Schemes</a> <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Everybody knows Uniform Resource Identifier (URI)1. As its name suggests, it’s a way to identify a resource (for instance a file or a phone number). It is often confused with Uniform Resource Locator (URL), which is actually a form of URI. A URL is a compact string representation for a resource available via the Internet2. Today I want to share with you the way I define another type of resources: network sockets. See RFC3986 “Uniform Resource Identifier (URI): Generic Syntax” ↩ Quoting RFC1738 “Uniform Resource Locators (URL)”, Section 1 “Introduction” ↩JWT scope claim compression using a bitmap2020-05-21T19:22:12+00:002020-05-21T19:22:12+00:00https://raphael.medaer.me/2020/05/21/scope-bitmap<p><a href="https://tools.ietf.org/html/rfc7519">JSON Web Tokens (JWT)</a> are often used in stateless authentication flows. Thanks to the signature, the server does not need anything else to verify the token validity.
The <code class="language-plaintext highlighter-rouge">scope</code> claim (<a href="https://tools.ietf.org/html/rfc8693#section-4.2">RFC8693 section 4.2</a>) contains <em>a space-separated list of scopes associated with the token</em>. The server can use it to check the application permissions.
Although this claim can quickly become heavy. The more scopes you have, the bigger your token is!
But JWT are meant to be a <a href="https://tools.ietf.org/html/rfc7519#section-1"><em>compact token format</em></a>…</p>
<p>Today I’m proud to present you an idea to compress scope list into a bitmap where one bit represents one scope.
<!--more--></p>
<h3 id="from-space-separated-list-to-bitmap">From space-separated list to bitmap</h3>
<p>The idea is quite simple. Instead of a huge string with all the scopes associated with the token, I associate each scope to a bit in a byte sequence. Each bit tells us if a scope is associated (or not) to the token.</p>
<p>For instance:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> 0 1 1 0 1 0 0 1 1 0 1 (...)
scope_a ──┘ │ │ │ │ │ │ │ │ │ │
scope_b ─────┘ │ │ │ │ │ │ │ │ │
scope_c ────────┘ ┆ ┆ ┆ ┆ ┆ ┆ ┆ ┆
(...) ┆ ┆ ┆ ┆ ┆ ┆ ┆
</code></pre></div></div>
<p>In this example, <code class="language-plaintext highlighter-rouge">scope_b</code> and <code class="language-plaintext highlighter-rouge">scope_c</code> are enabled, not <code class="language-plaintext highlighter-rouge">scope_a</code>.</p>
<p>As you already know, <em>byte-array</em> is not a JSON data type. But it’s not a big deal, we can encode these bytes in <a href="https://tools.ietf.org/html/rfc4648#section-4">Base 64</a>.</p>
<p>For a 3 bytes (max 24 scopes) claim, it would look like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
(...)
"b_scope": "NDIh"
}
</code></pre></div></div>
<h3 id="towards-a-new-claim">Towards a new claim</h3>
<p>I <del>don’t want to</del> can’t override the current <code class="language-plaintext highlighter-rouge">scope</code> claim specification. Even if technically it would not break the specification (because we still store a string inside), this is not a good practice IMO. Furthermore, in some cases you might want to have both a traditional <code class="language-plaintext highlighter-rouge">scope</code> claim and a bitmap scope claim.</p>
<p>That’s the reason why I decided to create a new claim: <code class="language-plaintext highlighter-rouge">b_scope</code>.</p>
<h3 id="where-and-how-to-define-the-scope-list-">Where and how to define the scope list ?</h3>
<p>Any resource provider is documenting the list of scopes available in its API. It’s defined either in a “human readable” documentation (text, html, pdf,…) or in a structured specification (<a href="https://tools.ietf.org/html/rfc8414">OAuth Authorization Server Metadata</a>, <a href="https://openid.net/specs/openid-connect-discovery-1_0.html">OpenID Connect Discovery</a>, OpenAPI,…).</p>
<p>For structured specifications my suggestion would be to keep the same ordered list and use the index of each scope as bit number.
The <a href="https://tools.ietf.org/html/rfc8414#section-2"><code class="language-plaintext highlighter-rouge">scopes_supported</code> field in OAuth Metadata</a> is a JSON array. Order of elements is preserved. However the <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#oauth-flow-object"><code class="language-plaintext highlighter-rouge">scopes</code> field in OpenAPI</a> is a map (JSON Object). We should not rely on the ordering of keys. Hopefuly the OpenAPI specification <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#specificationExtensions">allows extensions</a>. It means that we could define a <code class="language-plaintext highlighter-rouge">x-scopes-order</code> keyword which would list (JSON Array) the scopes with preserved ordering (like in OAuth Metadata).</p>
<h3 id="performances">Performances</h3>
<p>Is it a good idea from <strong>size</strong> point of view ?</p>
<p>Let say that you have 42 scopes in your API (<a href="https://developers.facebook.com/docs/facebook-login/permissions/">I use Facebook API as example</a>). It would require 6 bytes to store them in the bitmap. Because of Base 64 encoding we have to do <code class="language-plaintext highlighter-rouge">4 * n / 3</code> (+ rounded up mult 4) = 8 bytes. And to be fair, I added 2 bytes in claim name (prefix <code class="language-plaintext highlighter-rouge">b_</code> compared to <code class="language-plaintext highlighter-rouge">scope</code> claim) for a total of 10 bytes. The equivalent in size of scope list <code class="language-plaintext highlighter-rouge">email name</code>.</p>
<p>Is it a good idea from <strong>parsing</strong> (CPU load) point of view ?</p>
<p>Testing a byte sequence with binary operators is not CPU consuming. Compared to string manipulation and comparison it’s probably more performant.</p>
<h3 id="remaining-questions">Remaining questions</h3>
<h4 id="about-bit-numbering-endianness">About bit numbering (endianness)</h4>
<p><em>LSB</em> vs <em>MSB</em>. Currently I don’t know what’s the best option. I guess the answer should come from the following criteria:</p>
<ul>
<li>What’s the most efficient/performant ?
To be tested in multiple web-oriented languages (JS, Python, Ruby, Rust, Swift,…)</li>
<li>What’s the most “natural” in OAuth/JWT ecosystem ?</li>
</ul>
<h4 id="how-to-deprecate-a-scope-">How-to deprecate a scope ?</h4>
<p>You might want to deprecate a scope but you can’t remove this scope from the list otherwise it would shift all other following scopes.</p>
<p>In this case you will loose a bit. But if your tokens are limited in time, you could replace the bit scope with another after a deprecation period (longer than the max token lifetime).</p>
<p>But… do you often deprecate a scope ?</p>
<h4 id="and-the-dynamic-scopes-">And the “dynamic” scopes ?</h4>
<p>Some people are definining “dynamic” scopes. For instance: <code class="language-plaintext highlighter-rouge">pets.{id}.read</code>. I don’t think it’s a good practice. My understanding of scopes is to grant access to an API feature, not to the resource itself. In other words, IMHO you should have only one scope <code class="language-plaintext highlighter-rouge">pet.read</code> which allows application to read a pet. Your resources (aka content) ACL should not be in the scope itself.</p>
<p>Btw this bitmap scope mechanism is compatible with existing standard <code class="language-plaintext highlighter-rouge">scope</code> claim.</p>
<h4 id="could-we-use-this-b_scope-outside-of-jwt-">Could we use this <code class="language-plaintext highlighter-rouge">b_scope</code> outside of JWT ?</h4>
<p>Why not ? I didn’t analyze (yet) where and how we could use it but I guess it could be useful in some cases.
However I don’t see a big benefit to use it in OAuth (token and authorize) endpoints. These endpoints are called punctually while JWT is sent in each API request.</p>
<h3 id="follow-up">Follow up</h3>
<p>You might have questions or you just want to discuss about it. Please open issue in my blog repository or feel free to send me an email.</p>
<p>Here is some feedback from HN: <a href="https://news.ycombinator.com/item?id=23270052">https://news.ycombinator.com/item?id=23270052</a></p>JSON Web Tokens (JWT) are often used in stateless authentication flows. Thanks to the signature, the server does not need anything else to verify the token validity. The scope claim (RFC8693 section 4.2) contains a space-separated list of scopes associated with the token. The server can use it to check the application permissions. Although this claim can quickly become heavy. The more scopes you have, the bigger your token is! But JWT are meant to be a compact token format… Today I’m proud to present you an idea to compress scope list into a bitmap where one bit represents one scope.Manage multiple identities with Git2020-04-24T06:42:00+00:002020-04-24T06:42:00+00:00https://raphael.medaer.me/2020/04/24/manage-multiple-identities-with-git<p>I work on many different Git repositories. For each of them I have a particular email address and sometimes
a GPG key. Even the Git flow might be different; always <code class="language-plaintext highlighter-rouge">--no-ff</code> (or not), <code class="language-plaintext highlighter-rouge">pull --rebase</code> instead of merge,…
<!--more--></p>
<p>To deal with it I recently learned about <a href="https://git-scm.com/docs/git-config#_conditional_includes"><em>Conditional includes</em></a>.
It’s basically a way to include additional Git config files with a given condition. You have to configure it as a
new section in your Git repository.</p>
<p>In my case I organize projects and repositories with the following tree:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>documents
└── development
├── allocloud
│ └── (...) # ALLOcloud's repos
├── <project x>
│ └── (...) # Repos from project x
└── (...) # Other repos
</code></pre></div></div>
<p>For each project, I created a specific <code class="language-plaintext highlighter-rouge">.gitconfig</code> file that include in my main <code class="language-plaintext highlighter-rouge">~/.gitconfig</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[user]
name = Raphael Medaer
email = raphael@medaer.me
signingkey = D4D764423DCEA9FC90327C78FE29196052B47DF1
[commit]
gpgSign = true
[includeIf "gitdir:~/documents/development/allocloud/**"]
path = ~/documents/development/allocloud/.gitconfig
# (...)
</code></pre></div></div>I work on many different Git repositories. For each of them I have a particular email address and sometimes a GPG key. Even the Git flow might be different; always --no-ff (or not), pull --rebase instead of merge,…I hate capital letters!2020-04-23T19:31:00+00:002020-04-23T19:31:00+00:00https://raphael.medaer.me/2020/04/23/i-hate-capital-letters<p>I hate capital letters! At least in directory and file names. By default your home directory
has a few directories with first capital letter (<code class="language-plaintext highlighter-rouge">Documents</code>, <code class="language-plaintext highlighter-rouge">Music</code>, <code class="language-plaintext highlighter-rouge">Downloads</code>,…). Ok, it looks nice! But I always make mistake when I type capital letter in my terminal. Fortunately, all of this is configurable…
<!--more--></p>
<p>Thanks to <a href="https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html"><code class="language-plaintext highlighter-rouge">xdg</code></a>
and its command line tool: <code class="language-plaintext highlighter-rouge">xdg-user-dirs-update</code>! Man describes how to use <code class="language-plaintext highlighter-rouge">xdg-user-dirs-update --set NAME PATH</code> otherwise you can edit your global (or local) <code class="language-plaintext highlighter-rouge">xdg</code> configuration (on Debian <code class="language-plaintext highlighter-rouge">/etc/xdg/user-dirs.defaults</code>). Here is mine:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Default settings for user directories
#
# The values are relative pathnames from the home directory and
# will be translated on a per-path-element basis into the users locale
DESKTOP=desktop
DOWNLOAD=downloads
TEMPLATES=templates
PUBLICSHARE=public
DOCUMENTS=documents
MUSIC=music
PICTURES=pictures
VIDEOS=videos
</code></pre></div></div>I hate capital letters! At least in directory and file names. By default your home directory has a few directories with first capital letter (Documents, Music, Downloads,…). Ok, it looks nice! But I always make mistake when I type capital letter in my terminal. Fortunately, all of this is configurable…Cascade routing with AIOHTTP2019-07-03T05:30:00+00:002019-07-03T05:30:00+00:00https://raphael.medaer.me/2019/07/03/aiohttp-cascade-routing<p>A common routing use-case is to share a route URL pattern for multiple purposes. For instance GitHub is using <code class="language-plaintext highlighter-rouge">github.com/<something></code> for both users and organizations. Indeed the user and organization pages are different. A way to implement this is using a fallback mechanism called by some of us <em>cascade routing</em>.
<!--more-->
Meaning: fallback to next request handler if current one is not suitable. You’ll find this kind of behavior in popular front/backend framework <a href="https://github.com/angular/angular/pull/16416">such as Angular</a>. Actually it’s not yet available in Angular… 😒 and as far as I know not implemented in <a href="https://aiohttp.readthedocs.io/">AIOHTTP</a>. 😭</p>
<p><em>AN: I don’t compare AIOHTTP and Angular… obviously ! I just challenge the popularity of this feature.</em></p>
<p>In this short post I implement this mechanism with <a href="https://aiohttp.readthedocs.io/">AIOHTTP</a> !</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">NotHandledException</span><span class="p">(</span><span class="nb">Exception</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">CascadeRouter</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">handlers</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">_handlers</span> <span class="o">=</span> <span class="n">handlers</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">for</span> <span class="n">handler</span> <span class="ow">in</span> <span class="bp">self</span><span class="p">.</span><span class="n">_handlers</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="p">(</span><span class="k">await</span> <span class="n">handler</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">))</span>
<span class="k">except</span> <span class="n">NotHandledException</span><span class="p">:</span>
<span class="c1"># Fallback on next handler
</span> <span class="k">pass</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">HTTPNotFound</span><span class="p">()</span>
</code></pre></div></div>
<p>DONE ! Yes, that’s it ! Let’s analyze the code above…</p>
<p>First we define a new kind of <code class="language-plaintext highlighter-rouge">Exception</code> named <code class="language-plaintext highlighter-rouge">NotHandledException</code>. Users (aka developers) will have to raise this exception if they can’t deal with the given request.</p>
<p>The second class <code class="language-plaintext highlighter-rouge">CascadeRouter</code> is the router itself. <em>Ah bon ?!</em> It is instantiated with the ordered list of handlers to call. Thanks to <code class="language-plaintext highlighter-rouge">__call__</code> method, we can give this router to AIOHTTP which can call it like any handler.</p>
<p>Here is an example of usage:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="k">def</span> <span class="nf">handler_a</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">raise</span> <span class="n">NotHandledException</span><span class="p">()</span>
<span class="k">return</span> <span class="n">Response</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">'Hello from A'</span><span class="p">)</span>
<span class="k">async</span> <span class="k">def</span> <span class="nf">handler_b</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Response</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">'Hello from B'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">Application</span><span class="p">()</span>
<span class="n">app</span><span class="p">.</span><span class="n">add_routes</span><span class="p">([</span>
<span class="n">web</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="n">CascadeRouter</span><span class="p">([</span>
<span class="n">handler_a</span><span class="p">,</span>
<span class="n">handler_b</span><span class="p">,</span>
<span class="p">]))</span>
<span class="p">])</span>
</code></pre></div></div>
<p>In closing, we can discuss about the <code class="language-plaintext highlighter-rouge">for ... else</code> in <code class="language-plaintext highlighter-rouge">CascadeRouter#__call__</code>. Indeed it’s not mandatory however it’s IMHO more readable. Later we will be able to replace the <code class="language-plaintext highlighter-rouge">return</code> statement by a <code class="language-plaintext highlighter-rouge">break</code> and do some extra processing on handler result.</p>
<p>As usual, please <a href="https://github.com/rmedaer/rmedaer.github.io/issues">leave me a message in my blog issues</a> if you have any comment or advice.</p>
<p>R.</p>A common routing use-case is to share a route URL pattern for multiple purposes. For instance GitHub is using github.com/<something> for both users and organizations. Indeed the user and organization pages are different. A way to implement this is using a fallback mechanism called by some of us cascade routing.Open RXVT terminal in current directory2019-06-21T10:00:00+00:002019-06-21T10:00:00+00:00https://raphael.medaer.me/2019/06/21/urxvt-open-current-wd<p>Two years ago I switched from terminator to URXVT. This is now my day to day terminal emulator. URXVT is the unicode version of RXVT started long time ago by Rob Nation. Today I’ll show you how I fixed one of the very missing useful feature: <strong>open a new terminal in current working directory</strong>. By <em>“current working directory”</em> I mean <em>“the working directory of the shell in the most recent focused terminal”</em>.
<!--more--></p>
<p>This is actually a frequently asked question. However I didn’t find answer which convinced me. Here are some threads talking about:</p>
<ul>
<li><a href="https://www.reddit.com/r/archlinux/comments/7km09z/how_to_open_new_urxvt_window_in_the_same/">on reddit: <em>“How to open new urxvt window in the same directory as focused (…)”</em></a></li>
<li><a href="https://superuser.com/questions/759294/how-to-open-new-terminal-in-current-directory">on stack superuser: <em>“How to open new terminal in current directory?”</em></a></li>
<li><a href="https://faq.i3wm.org/question/150/how-to-launch-a-terminal-from-here/%3C/p%3E.html">on i3wm: <em>“How to launch a terminal “from here”?”</em></a></li>
</ul>
<p>The proposed answers are IMHO weak scripts or hack playing with X11. I want a structural solution and thanks to RXVT plugins we can implement it … in Perl 😒.
But first here are my requirements:</p>
<ul>
<li>MUST be optional; I still have to open terminal in home directory</li>
<li>MUST work without having to parse window title or other fancy X11 trick</li>
<li>SHOULD work with multi-user and multi-display</li>
</ul>
<p>With the following URXVT Perl extension(s), I hope to fulfill this feature once and for all.</p>
<p>I implemented two new URXVT extensions: <code class="language-plaintext highlighter-rouge">remember-last-dir</code> and <code class="language-plaintext highlighter-rouge">open-last-dir</code>.</p>
<p><code class="language-plaintext highlighter-rouge">remember-last-dir</code> keeps in shared memory the shell PID of last focused terminal. Currently the IPC key is using only the user ID. Not the display ID. However it could be a nice improvement. This extension must always be enabled.</p>
<div class="language-pl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/bin/perl</span>
<span class="k">use</span> <span class="nn">IPC::</span><span class="nv">Shareable</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">$glue</span> <span class="o">=</span> <span class="nb">sprintf</span> <span class="p">'</span><span class="s1">urxvt-last-dir-%d</span><span class="p">',</span> <span class="vg">$<</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">%options</span> <span class="o">=</span> <span class="p">(</span>
<span class="s">create</span> <span class="o">=></span> <span class="p">'</span><span class="s1">yes</span><span class="p">',</span>
<span class="s">exclusive</span> <span class="o">=></span> <span class="mi">0</span><span class="p">,</span>
<span class="s">mode</span> <span class="o">=></span> <span class="mo">0700</span><span class="p">,</span>
<span class="s">destroy</span> <span class="o">=></span> <span class="p">'</span><span class="s1">no</span><span class="p">',</span>
<span class="p">);</span>
<span class="k">sub </span><span class="nf">on_focus_in</span> <span class="p">{</span>
<span class="k">my</span> <span class="p">(</span><span class="nv">$self</span><span class="p">)</span> <span class="o">=</span> <span class="nv">@_</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">$pid</span><span class="p">;</span>
<span class="nb">tie</span> <span class="nv">$pid</span><span class="p">,</span> <span class="p">'</span><span class="s1">IPC::Shareable</span><span class="p">',</span> <span class="nv">$glue</span><span class="p">,</span> <span class="p">{</span> <span class="nv">%options</span> <span class="p">};</span>
<span class="nv">$pid</span> <span class="o">=</span> <span class="nv">$self</span><span class="o">-></span><span class="p">{</span><span class="nv">shell_pid</span><span class="p">};</span>
<span class="p">}</span>
<span class="k">sub </span><span class="nf">on_child_start</span> <span class="p">{</span>
<span class="k">my</span><span class="p">(</span><span class="nv">$self</span><span class="p">,</span> <span class="nv">$pid</span><span class="p">)</span> <span class="o">=</span> <span class="nv">@_</span><span class="p">;</span>
<span class="nv">$self</span><span class="o">-></span><span class="p">{</span><span class="nv">shell_pid</span><span class="p">}</span> <span class="o">=</span> <span class="nv">$pid</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">open-last-dir</code> is initializing the terminal with the working directory of current PID in shared memory. This extension is optionally launched from URXVT command line: <code class="language-plaintext highlighter-rouge">urxvt -pe open-last-dir</code>.</p>
<div class="language-pl highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/bin/perl</span>
<span class="k">use</span> <span class="nn">IPC::</span><span class="nv">Shareable</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">$glue</span> <span class="o">=</span> <span class="nb">sprintf</span> <span class="p">'</span><span class="s1">urxvt-last-dir-%d</span><span class="p">',</span> <span class="vg">$<</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">%options</span> <span class="o">=</span> <span class="p">(</span>
<span class="s">create</span> <span class="o">=></span> <span class="p">'</span><span class="s1">yes</span><span class="p">',</span>
<span class="s">exclusive</span> <span class="o">=></span> <span class="mi">0</span><span class="p">,</span>
<span class="s">mode</span> <span class="o">=></span> <span class="mo">0700</span><span class="p">,</span>
<span class="s">destroy</span> <span class="o">=></span> <span class="p">'</span><span class="s1">no</span><span class="p">',</span>
<span class="p">);</span>
<span class="k">sub </span><span class="nf">on_init</span> <span class="p">{</span>
<span class="k">my</span> <span class="p">(</span><span class="nv">$self</span><span class="p">)</span> <span class="o">=</span> <span class="nv">@_</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">$pid</span><span class="p">;</span>
<span class="nb">tie</span> <span class="nv">$pid</span><span class="p">,</span> <span class="p">'</span><span class="s1">IPC::Shareable</span><span class="p">',</span> <span class="nv">$glue</span><span class="p">,</span> <span class="p">{</span> <span class="nv">%options</span> <span class="p">};</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">defined</span> <span class="nv">$pid</span> <span class="ow">and</span> <span class="nv">$pid</span> <span class="ow">ne</span> <span class="p">"")</span> <span class="p">{</span>
<span class="k">my</span> <span class="nv">$link</span> <span class="o">=</span> <span class="nb">sprintf</span> <span class="p">"</span><span class="s2">/proc/%d/cwd</span><span class="p">",</span> <span class="nv">$pid</span><span class="p">;</span>
<span class="k">my</span> <span class="nv">$wd</span> <span class="o">=</span> <span class="nb">readlink</span> <span class="nv">$link</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="o">-</span><span class="nv">e</span> <span class="nv">$wd</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$self</span><span class="o">-></span><span class="nv">resource</span><span class="p">("</span><span class="s2">chdir</span><span class="p">",</span> <span class="nv">$wd</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>I guess we could improve this code. This is here a quick dump of my setup. Please don’t blame me for missing error handling, code factoring, … If you want to contribute or help me going further with this extension, please <a href="https://github.com/rmedaer/rmedaer.github.io/issues">leave me a message in my blog issues</a>.</p>
<p>Thanks for reading,</p>
<p>R.</p>Two years ago I switched from terminator to URXVT. This is now my day to day terminal emulator. URXVT is the unicode version of RXVT started long time ago by Rob Nation. Today I’ll show you how I fixed one of the very missing useful feature: open a new terminal in current working directory. By “current working directory” I mean “the working directory of the shell in the most recent focused terminal”.