tag:blogger.com,1999:blog-3475826180450554102024-03-14T13:03:49.339+00:00Surguta personal blog of Dimitri John LedkovDimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.comBlogger36125tag:blogger.com,1999:blog-347582618045055410.post-65904060168979290992024-01-25T18:01:00.000+00:002024-01-25T18:01:15.852+00:00Ubuntu Livepatch service now supports over 60 different kernels<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgECGwNOL2qcUgUIbptrFlOV-rAL4ehu3xDLwf2sk_MUkI4x6O1Y4WcRs9pueMk-pxvXfdPG-kTFV8if3OLCwQhimAVsZ9gaod02j-jwsP-Cr3bpf2Jx_oH1k-WfIo4GV4iy5Tr_uQ0StaR2F46r7RDzFJx3SXFs7sComSKPFkOJRhlwAVBGXOCaJKGGxE/s1024/_d0db3ba5-728b-4c40-b8d1-fa6d434b7667.jpeg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1024" data-original-width="1024" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgECGwNOL2qcUgUIbptrFlOV-rAL4ehu3xDLwf2sk_MUkI4x6O1Y4WcRs9pueMk-pxvXfdPG-kTFV8if3OLCwQhimAVsZ9gaod02j-jwsP-Cr3bpf2Jx_oH1k-WfIo4GV4iy5Tr_uQ0StaR2F46r7RDzFJx3SXFs7sComSKPFkOJRhlwAVBGXOCaJKGGxE/w320-h320/_d0db3ba5-728b-4c40-b8d1-fa6d434b7667.jpeg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Linux kernel getting a livepatch whilst running a marathon. Generated with AI.</td></tr></tbody></table><p><a href="https://ubuntu.com/security/livepatch" target="_blank">Livepatch service</a> eliminates the need for unplanned maintenance windows for high and critical severity kernel vulnerabilities by patching the Linux kernel while the system runs. Originally the service <a href="https://lists.ubuntu.com/archives/ubuntu-announce/2016-October/000214.html">launched</a> in 2016 with just a single kernel flavour supported.</p><p>Over the years, additional kernels were added: new LTS releases, ESM kernels, Public Cloud kernels, and most recently HWE kernels too.</p><p>Recently livepatch support was expanded for FIPS compliant kernels, Public cloud FIPS compliant kernels, and as well IBM Z (mainframe) kernels. Bringing the total of kernel flavours support to over 60 distinct kernel flavours supported in parallel. The table of <a href="https://ubuntu.com/security/livepatch/docs/livepatch/reference/kernels">supported kernels</a> in the documentation lists the supported kernel flavours ABIs, the duration of individual build's support window, supported architectures, and the Ubuntu release. This work was only possible thanks to the collaboration with the Ubuntu Certified Public Cloud team, engineers at IBM for IBM Z (s390x) support, Ubuntu Pro team, Livepatch server & client teams.</p><p>It is a great milestone, and I personally enjoy seeing the non-intrusive popup on my Ubuntu Desktop that a kernel livepatch was applied to my running system. I do enable <a href="https://ubuntu.com/pro">Ubuntu Pro</a> on my personal laptop thanks to the free Ubuntu Pro subscription for individuals.</p><p>What's next? The next frontier is supporting ARM64 kernels. The Canonical kernel team has completed the gap analysis to start supporting Livepatch Service for ARM64. Upstream Linux requires development work on the consistency model to fully support livepatch on ARM64 processors. Livepatch code changes are applied on a per-task basis, when the task is deemed safe to switch over. This safety check depends mostly on kernel stacktraces. For these checks, CONFIG_HAVE_RELIABLE_STACKTRACE needs to be available in the upstream ARM64 kernel. (see <a href="https://www.kernel.org/doc/html/latest/livepatch/livepatch.html#adding-consistency-model-support-to-new-architectures" target="_blank">The Linux Kernel Documentation</a>). There are preliminary patches that enable reliable stacktraces on ARM64, <a href="https://github.com/dynup/kpatch/pull/1302#issue-1375125587" target="_blank">however these turned out to be problematic</a> as there are lots of <a href="https://lore.kernel.org/all/20220707150134.4614-1-madvenka@linux.microsoft.com/#r">fix revisions</a> that came after the initial patchset that AWS ships with 5.10. This is a call for help from any interested parties. If you have engineering resources and are interested in bringing Livepatch Service to your ARM64 platforms, please reach out to the Canonical Kernel team on the public Ubuntu Matrix, Discourse, and mailing list. If you want to chat in person, see you at <a href="https://fosdem.org/2024/">FOSDEM</a> next weekend.</p>Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-69330142249737316192023-11-16T10:45:00.000+00:002023-11-16T10:45:08.868+00:00Ubuntu 23.10 significantly reduces the installed kernel footprint<p><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; white-space-collapse: preserve;"><br /></span></p><p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjn4rImnreYgG1NES8wXyPT2ML-m8NJ7y5FrxF7Xek7Nni0ZF0F4OLx567s-zeMaYbxiOU-K8KDkUduWeX55fHVWvI30YI_S5o7Q6Y69xtVS25Uis7FI1SIGc7RtKBAgwPbn2RsB30YOv8BBSr4a1fVEeZROSbD2ga7EJZQIhjmpXP02ubhRPZkGH5fVUk/s1280/pexels-pixabay-372796.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="960" data-original-width="1280" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjn4rImnreYgG1NES8wXyPT2ML-m8NJ7y5FrxF7Xek7Nni0ZF0F4OLx567s-zeMaYbxiOU-K8KDkUduWeX55fHVWvI30YI_S5o7Q6Y69xtVS25Uis7FI1SIGc7RtKBAgwPbn2RsB30YOv8BBSr4a1fVEeZROSbD2ga7EJZQIhjmpXP02ubhRPZkGH5fVUk/w400-h300/pexels-pixabay-372796.jpg" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-family: PlusJakartaSans, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Cantarell, "Helvetica Neue", Ubuntu, sans-serif; font-size: 14px; text-align: left; white-space: pre;">Photo by <a href="https://www.pexels.com/photo/metal-pippings-with-pressure-gauge-372796/" target="_blank">Pixabay</a></span></td></tr></tbody></table></p><p><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; white-space-collapse: preserve;">Ubuntu systems typically have up to 3 kernels installed, before they are auto-removed by apt on classic installs. Historically the installation was optimized for metered download size only. However, kernel size growth and usage no longer warrant such optimizations. During the 23.10 Mantic Minatour cycle, I led a coordinated effort across multiple teams to implement lots of optimizations that together achieved unprecedented install footprint improvements.</span></p><span id="docs-internal-guid-66c11ab4-7fff-5998-0d03-a26cf6ea14ef"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Given a typical install of 3 generic kernel ABIs in the default configuration on a regular-sized VM (2 CPU cores 8GB of RAM) the following metrics are achieved in Ubuntu 23.10 versus Ubuntu 22.04 LTS:</span></p><ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">2x less disk space used (1,417MB vs 2,940MB, including initrd)</span></p></li><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">3x less peak RAM usage for the initrd boot (68MB vs 204MB)</span></p></li><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">0.5x increase in download size (949MB vs 600MB)</span></p></li><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">2.5x faster initrd generation (4.5s vs 11.3s)</span></p></li><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">approximately the same total time (103s vs 98s, hardware dependent)</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">For minimal cloud images that do not install either linux-firmware or modules extra the numbers are:</span></p><ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">1.3x less disk space used (548MB vs 742MB)</span></p></li><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">2.2x less peak RAM usage for initrd boot (27MB vs 62MB)</span></p></li><li aria-level="1" dir="ltr" style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">0.4x increase in download size (207MB vs 146MB)</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Hopefully, the compromise of download size, relative to the disk space & initrd savings is a win for the majority of platforms and use cases. For users on extremely expensive and metered connections, the likely best saving is to receive air-gapped updates or skip updates.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">This was achieved by precompressing kernel modules & firmware files with the maximum level of Zstd compression at package build time; making actual .deb files uncompressed; assembling the initrd using split cpio archives - uncompressed for the pre-compressed files, whilst compressing only the userspace portions of the initrd; enabling in-kernel module decompression support with matching kmod; fixing bugs in all of the above, and landing all of these things in time for the feature freeze. Whilst leveraging the experience and some of the design choices implementations we have already been shipping on Ubuntu Core. Some of these changes are backported to Jammy, but only enough to support smooth upgrades to Mantic and later. Complete gains are only possible to experience on Mantic and later.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The discovered bugs in kernel module loading code likely affect systems that use LoadPin LSM with kernel space module uncompression as used on ChromeOS systems. Hopefully, Kees Cook or other ChromeOS developers pick up the kernel fixes from the stable trees. Or you know, just use Ubuntu kernels as they do get fixes and features like these first.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The team that designed and delivered these changes is large: Benjamin Drung, Andrea Righi, Juerg Haefliger, Julian Andres Klode, Steve Langasek, Michael Hudson-Doyle, Robert Kratky, Adrien Nader, Tim Gardner, Roxana Nicolescu - and myself Dimitri John Ledkov ensuring the most optimal solution is implemented, everything lands on time, and even implementing portions of the final solution.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Hi, It's me, I am a Staff Engineer at Canonical and we are hiring </span><a href="https://canonical.com/careers" style="text-decoration-line: none;" target="_blank"><span style="color: #4a6ee0; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://canonical.com/careers</span></a><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #0e101a; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Lots of additional technical details and benchmarks on a huge range of diverse hardware and architectures, and bikeshedding all the things below:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"></p><ul style="text-align: left;"><li><a href="https://lists.ubuntu.com/archives/ubuntu-devel/2023-July/thread.html#42652" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://lists.ubuntu.com/archives/ubuntu-devel/2023-July/thread.html#42652</span></a></li><li><a href="https://lists.ubuntu.com/archives/kernel-team/2023-July/thread.html#141412" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://lists.ubuntu.com/archives/kernel-team/2023-July/thread.html#141412</span></a></li><li><a href="https://lists.ubuntu.com/archives/ubuntu-devel/2023-July/thread.html#42707" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://lists.ubuntu.com/archives/ubuntu-devel/2023-July/thread.html#42707</span></a></li><li><a href="https://discourse.ubuntu.com/t/reduce-initramfs-size-and-speed-up-the-generation-in-ubuntu-23-10/38972" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://discourse.ubuntu.com/t/reduce-initramfs-size-and-speed-up-the-generation-in-ubuntu-23-10/38972</span></a></li><li><a href="https://lore.kernel.org/all/20230830155820.138178-1-andrea.righi@canonical.com/" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://lore.kernel.org/all/20230830155820.138178-1-andrea.righi@canonical.com/</span></a></li><li><a href="https://lore.kernel.org/all/20230829123808.325202-1-andrea.righi@canonical.com/" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://lore.kernel.org/all/20230829123808.325202-1-andrea.righi@canonical.com/</span></a></li><li><a href="https://facebook.github.io/zstd/" style="text-decoration-line: none;" target="_blank"><span style="color: #1155cc; font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://facebook.github.io/zstd/</span></a></li></ul><p></p>For questions and comments please post to Kernel section on <a href="https://discourse.ubuntu.com/c/kernel/108" target="_blank">Ubuntu Discourse</a>.<br /><br /><br /><br /></span>Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.comtag:blogger.com,1999:blog-347582618045055410.post-78105474909142360212019-08-30T16:42:00.000+01:002019-08-30T16:42:38.628+01:00How to disable TLS 1.0 and TLS 1.1 on Ubuntu<div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ2ibUiW30SEURIxCIfL08msgCcWcOyJNSHDvpcF7BnYjVsUARHLkaCmdq1QKTevqne2gMBiTyb1anvjvTD4aNl8gonrXTD4vOXf4jJIcdMd6lDHOHAYnZh_1wLCwEdUkYvSPZtn_aWS8/s1600/tls-1.2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1366" data-original-width="1600" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ2ibUiW30SEURIxCIfL08msgCcWcOyJNSHDvpcF7BnYjVsUARHLkaCmdq1QKTevqne2gMBiTyb1anvjvTD4aNl8gonrXTD4vOXf4jJIcdMd6lDHOHAYnZh_1wLCwEdUkYvSPZtn_aWS8/s320/tls-1.2.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Example of website that only supports TLS v1.0, which is rejected by the client</td></tr>
</tbody></table>
<h3 style="text-align: left;">
Overivew</h3>
TLS v1.3 is the latest standard for secure communication over the internet. It is widely supported by desktops, servers and mobile phones. Recently Ubuntu 18.04 LTS received OpenSSL 1.1.1 update bringing the ability to potentially establish TLS v1.3 connections on the latest Ubuntu LTS release. <a href="https://www.ssllabs.com/ssl-pulse/" target="_blank">Qualys SSL Labs Pulse</a> report shows more than 15% adoption of TLS v1.3. It really is time to migrate from TLS v1.0 and TLS v1.1.<br />
<br />
As announced on the 15th of October 2018 <a href="https://webkit.org/blog/8462/deprecation-of-legacy-tls-1-0-and-1-1-versions/" target="_blank">Apple</a>, <a href="https://security.googleblog.com/2018/10/modernizing-transport-security.html" target="_blank">Google</a>, and <a href="https://blogs.windows.com/msedgedev/2018/10/15/modernizing-tls-edge-ie11/" target="_blank">Microsoft</a> will disable TLS v1.0 and TLS v1.1 support by default and thus require TLS v1.2 to be supported by all clients and servers. Similarly, Ubuntu 20.04 LTS will also require TLS v1.2 as the minimum TLS version as well.<br />
<br />
To prepare for the move to TLS v1.2, it is a good idea to disable TLS v1.0 and TLS v1.1 on your local systems and start observing and reporting any websites, systems and applications that do not support TLS v1.2.<br />
<h3 style="text-align: left;">
How to disable TLS v1.0 and TLS v1.1 in Google Chrome on Ubuntu</h3>
<div>
<ol style="text-align: left;">
<li>Create policy directory<br /><blockquote class="tr_bq">
sudo mkdir -p /etc/opt/chrome/policies/managed</blockquote>
</li>
<li>Create /etc/opt/chrome/policies/managed/mintlsver.json with<br /><blockquote class="tr_bq">
{<br />
"SSLVersionMin" : "tls1.2"<br />
} </blockquote>
</li>
</ol>
</div>
<h3>
How to disable TLS v1.0 and TLS v1.1 in Firefox on Ubuntu</h3>
<div>
<ol style="text-align: left;">
<li>Navigate to <b>about:config</b> in the URL bar</li>
<li>Search for <b>security.tls.version.min</b> setting</li>
<li>Set it to 3, which stand for minimum TLS v1.2</li>
</ol>
<div>
<h3 style="text-align: left;">
How to disable TLS v1.0 and TLS v1.1 in OpenSSL</h3>
</div>
</div>
<div>
<ol style="text-align: left;">
<li>Edit /etc/ssl/openssl.cnf</li>
<li>After oid_section stanza add<br /><blockquote class="tr_bq">
# System default<br />
openssl_conf = default_conf</blockquote>
</li>
<li>After oid_section stanza add<br /><blockquote class="tr_bq">
[default_conf]<br />
ssl_conf = ssl_sect<br />
<br />
[ssl_sect]<br />
system_default = system_default_sect<br />
<br />
[system_default_sect]<br />
MinProtocol = TLSv1.2<br />
CipherString = DEFAULT@SECLEVEL=2</blockquote>
</li>
<li> Save the file</li>
</ol>
</div>
<div>
</div>
<h3 style="text-align: left;">
How to disable TLS v1.0 and TLS v1.1 in GnuTLS</h3>
<div>
<ol>
<li>Create config directory<br /><blockquote class="tr_bq">
sudo mkdir -p /etc/gnutls/</blockquote>
</li>
<li>Create /etc/gnutls/default-priorities with<br /><blockquote class="tr_bq">
SYSTEM=SECURE192:-VERS-ALL:+VERS-TLS1.3:+VERS-TLS1.2 </blockquote>
</li>
</ol>
</div>
After performing above tasks most common applications will use TLS v1.2+<br />
<br />
I have set these defaults on my systems, and I occasionally hit websites that only support TLS v1.0 and I report them. Have you found any websites and systems you use that do not support TLS v1.2 yet?</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com6tag:blogger.com,1999:blog-347582618045055410.post-5413962084199255152019-02-13T23:09:00.000+00:002019-02-13T23:09:59.843+00:00Encrypt all the things<div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://imgs.xkcd.com/comics/security.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="274" data-original-width="448" src="https://imgs.xkcd.com/comics/security.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://xkcd.com/538/" target="_blank">xkcd #538: Security</a></td></tr>
</tbody></table>
Went into blogger settings and enabled TLS on my custom domain blogger blog. So it is now finally a <a href="https://blog.surgut.co.uk/">https://blog.surgut.co.uk</a> However, I do use feedburner and syndicate that to the planet. I am not sure if that is end-to-end TLS connections, thus I will look into removing feedburner between my blog and the ubuntu/debian planets. My experience with changing feeds in the planets is that I end up spamming everyone. I wonder, if I should make a new tag and add that one, and add both feeds to the planet config to avoid spamming old posts.<br />
<br />
Next up went into gandi LiveDNS platform and enabled DNSSEC on my domain. It propagated quite quickly, but I believe my domain is now correctly signed with DNSSEC stuff. Next up I guess, is to fix DNSSEC with captive portals. I guess what we really want to have on "wifi" like devices, is to first connect to wifi and not set it as default route. Perform captive portal check, potentially with a reduced DNS server capabilities (ie. no EDNS, no DNSSEC, etc) and only route traffic to the captive portal to authenticate. Once past the captive portal, test and upgrade connectivity to have DNSSEC on. In the cloud, and on the wired connections, I'd expect that DNSSEC should just work, and if it does we should be enforcing DNSSEC validation by default.<br />
<br />
So I'll start enforcing DNSSEC on my laptop I think, and will start reporting issues to all of the UK banks if they dare not to have DNSSEC. If I managed to do it, on my own domain, so should they!<br />
<br />
Now I need to publish CAA Records to indicate that my sites are supposed to be protected by Let's Encrypt certificates only, to prevent anybody else issuing certificates for my sites and clients trusting them.<br />
<br />
I think I think I want to publish SSHFP records for the servers I care about, such that I could potentially use those to trust the fingerprints. Also at the <a href="https://fosdem.org/2019/schedule/event/dns_getdns_local_validation/" target="_blank">FOSDEM getdns</a> talk it was mentioned that openssh might not be verifying these by default and/or need additional settings pointing at the anchor. Will need to dig into that, to see if I need to modify something about this. It did sound odd.<br />
<br />
Generated 4k RSA subkeys for my main key. Previously I was using 2k RSA keys, but since I got a new yubikey that supports 4k keys I am upgrading to that. I use yubikey's OpenGPG for my signing, encryption, and authentication subkeys - meaning for ssh too. Which I had to remember how to use `gpg --with-keygrip -k` to add the right "keygrip" to `~/.gnupg/sshcontrol` file to get the new subkey available in the ssh agent. Also it seems like the order of keygrips in sshcontrol file matters. Updating new ssh key in all the places is not fun I think I did github, salsa and launchpad at the moment. But still need to push the keys onto the many of the installed systems.<br />
<br />
Tried to use FIDO2 passwordless login for Windows 10, only to find out that my Dell XPS appears to be incompatible with it as it seems that my laptop does not have TPM. Oh well, I guess I need to upgrade my laptop to have a TPM2 chip such that I can have self-unlocking encrypted drives, and like OTP token displayed on boot and the like as was presented at <a href="https://fosdem.org/2019/schedule/event/tpm2/" target="_blank">this FOSDEM</a> talk.<br />
<br />
Now that <a href="https://tracker.debian.org/news/1028794/accepted-cryptsetup-2210-1-source-into-unstable/" target="_blank">cryptsetup 2.1.0</a> is out and is in Debian and Ubuntu, I guess it's time to reinstall and re-encrypt my laptop, to migrate from LUKS1 to LUKS2. It has a bigger header, so obviously so much better!<br />
<br />
Changing phone soon, so will need to regenerate all of the OTP tokens. <b>*sigh* </b>Does anyone backup all the QR codes for them, to quickly re-enroll all the things?<br />
<br />
BTW I gave a talk about <a href="https://fosdem.org/2019/schedule/event/dns_systemd_resolved/" target="_blank">systemd-resolved at FOSDEM</a>. People didn't like that we do not enable/enforce DNS over TLS, or DNS over HTTPS, or DNSSEC by default. At least, people seemed happy about not leaking queries. But not happy again about caching.<br />
<br />
I feel safe.<br />
<br />
ps. funny how xkcd uses 2k RSA, not 4k.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com3London, UK51.5073509 -0.1277582999999822351.1912379 -0.77320529999998222 51.8234639 0.51768870000001777tag:blogger.com,1999:blog-347582618045055410.post-85070642890514325622018-02-06T15:25:00.002+00:002018-02-06T15:25:48.135+00:00Ubuntu Snowsports & Friends Team<div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYI6ICxbgV2eZp8QypOA5oEdFrwdFflPWmZTHMoX6w7DogqwqvdpO0LFS1eQSNSoPR5CyrJe3LLzEk-DEy-uwcTgNM9_UTuMmt-b117AQApRqAYaoOnP8TBNFb3WSTgzpL79hWVbRohOE/s1600/pictogram-cloud-orange.svg192.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="192" data-original-width="192" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYI6ICxbgV2eZp8QypOA5oEdFrwdFflPWmZTHMoX6w7DogqwqvdpO0LFS1eQSNSoPR5CyrJe3LLzEk-DEy-uwcTgNM9_UTuMmt-b117AQApRqAYaoOnP8TBNFb3WSTgzpL79hWVbRohOE/s1600/pictogram-cloud-orange.svg192.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Ubuntu Snowsports and Friends Team</td></tr>
</tbody></table>
<br />
After talking to a bunch of people, I've realized that a lot of free & open source, debian / ubuntu, etc people do ski or snowboard. So I have this crazy idea, that maybe we can get enough people together to form a social team on Launchpad.<br />
<br />
And maybe if we have enough people there, to possibly try to organize a ski trip with or without conference talks. Kind of like a team building meetup / community event / UDS - Ubuntu Developer Snowsports trip, or maybe an Ubucon Snow.<br />
<br />
So here we go - please consider joining <a href="https://launchpad.net/~ubuntu-snowsports">https://launchpad.net/~ubuntu-snowsports</a> team, join the mailing list there, and/or hop onto IRC to join #ubuntu-snow on freenode.<br />
<br />
I hope we can get more members than <a href="https://launchpad.net/~ubuntu-cyclists">https://launchpad.net/~ubuntu-cyclists</a></div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-20273446067179205502017-12-15T09:09:00.002+00:002017-12-15T09:09:37.748+00:00What does FCC Net Neutrality repeal mean to you?<div dir="ltr" style="text-align: left;" trbidi="on">
<center>
<div dir="ltr" style="background-color: #f1f1f1; border-color: black; border-radius: 30px; border: 2px solid; padding: 10px; text-align: left; width: 400px;" trbidi="on">
<h1 style="font-family: "arial" , sans-serif; font-size: 20px; font-weight: bold; line-height: 1.2em;">
Sorry, the web page you have requested is not available through your internet connection.</h1>
<h1 style="text-align: center;">
<div style="font-family: Arial, sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; margin-bottom: 18.08px; text-align: left;">
We have received an order from the Courts requiring us to prevent access to this site in order to help protect against Lex Julia Majestatis infridgement.</div>
<hr style="font-family: Arial, sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; text-align: left;" />
<div style="font-family: "arial" , sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; margin-top: 18.08px; margin-top: 18px; text-align: left;">
If you are a home broadband customer, for more information on why certain web pages are blocked, please click <a href="https://www.eff.org/deeplinks/content-blocking" style="color: #cc0000; text-decoration: none;" target="_blank" title="Home broadband">here</a>.</div>
<div style="font-family: "arial" , sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; margin-top: 18px; text-align: left;">
If you are a business customer, or are trying to view this page through your company's internet connection, please click <a href="https://www.eff.org/deeplinks/content-blocking" style="color: #cc0000; text-decoration: none;" target="_blank" title="Business">here</a>.
<br />
<div style="color: red; font-family: "arial" , sans-serif; font-size: 80px; font-weight: normal; line-height: 18.079999923706055px; margin-bottom: 18.08px; margin-top: 18px; text-align: left;">
∞
</div>
</div>
</h1>
</div>
</center>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-29719253570959332562017-10-03T14:27:00.000+01:002017-10-03T14:27:53.642+01:00An interesting bug - network-manager, glibc, dpkg-shlibdeps, systemd, and finally binutils<div dir="ltr" style="text-align: left;" trbidi="on">
Not so long ago I went to effectively recompile <a href="https://launchpad.net/ubuntu/+source/network-manager/1.8.2-1ubuntu6" target="_blank">NetworkManager</a> and fix up minor bug in it. It built fine across all architectures, was considered to be installable etc. And I was expecting it to just migrate across. At the time, glibc was at 2.26 in artful-proposed and NetworkManager was built against it. However release pocket was at glibc 2.24. In Ubuntu we have a <a href="https://wiki.ubuntu.com/ProposedMigration" target="_blank">ProposedMigration</a> process in place which ensures that newly built packages do not regress in the number of architectures built for; installable on; and do not regress themselves or any reverse dependencies at runtime.<br />
<br />
Thus before my build of NetworkManager was considered for migration, it was tested in the release pocket against packages in the release pocket. Specifically, since package metadata only requires glibc 2.17 NetworkManager was tested against glibc currently in the release pocket, which should just work fine....<br />
<blockquote class="tr_bq" style="white-space: pre-wrap; word-wrap: break-word;">
autopkgtest [21:47:38]: test nm: [-----------------------<br />test_auto_ip4 (__main__.ColdplugEthernet)<br />ethernet: auto-connection, IPv4 ... FAIL
----- NetworkManager.log -----<br />NetworkManager: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by NetworkManager)</blockquote>
<div>
At first I only saw failing tests, which I thought is transient failure. Thus they were retried a few <a href="http://autopkgtest.ubuntu.com/packages/network-manager/artful/amd64" target="_blank">times</a>. Then I looked at the <a href="https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-artful/artful/amd64/n/network-manager/20170906_215320_fc70c@/log.gz" target="_blank">autopkgtest log</a> and saw above error messages. Perplexed, I have started a lxd container with ubuntu artful, enabled proposed and installed just network-manager from artful-proposed and indeed a simple `NetworkManager --help` failed with above error from linker.</div>
<div>
<br /></div>
<div>
I am too young to know what dependency-hell means, since ever since I used Linux (Ubuntu 7.04) all glibc symbols were versioned, and <a href="http://manpages.ubuntu.com/manpages/xenial/en/man1/dpkg-shlibdeps.1.html" target="_blank">dpkg-shlibdeps</a> would generate correct minimum dependencies for a package. Alas in this case readelf confirmed that indeed /usr/sbin/NetworkManager requires 2.25 and dpkg depends is >= 2.17.</div>
<div>
<br /></div>
<div>
Further reading readelf output I checked that all of the glibc symbols used are 2.17 or lower, and only the "Version needs section '.gnu.version_r'" referenced GLIBC_2.25 symbol. Inspecting dpkg-shlibdeps code I noticed that it does not parse that section and only searches through the dynamic symbols used to establish the minimum required version.</div>
<div>
<br /></div>
<div>
Things started to smell fishy. On one hand, I trust dpkg-shlibdeps to generate the right dependencies. On the other hand I also trust linker to not tell lies either. Hence I opened a Debian BTS <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=874585" target="_blank">bug report</a> about this issue.</div>
<div>
<br /></div>
<div>
At this point, I really wanted to figure out where the reference to 2.25 comes from. Clearly it was not from any private symbols as then the reference would be on 2.26. Checking glibc abi lists I found there were only a handful of symbols marked as 2.25</div>
<div>
<blockquote class="tr_bq">
$ grep 2.25 ./sysdeps/unix/sysv/linux/x86_64/64/libc.abilist<br />GLIBC_2.25 GLIBC_2.25 A<br />GLIBC_2.25 __explicit_bzero_chk F<br />GLIBC_2.25 explicit_bzero F<br />GLIBC_2.25 getentropy F<br />GLIBC_2.25 getrandom F<br />GLIBC_2.25 strfromd F<br />GLIBC_2.25 strfromf F<br />GLIBC_2.25 strfroml F</blockquote>
</div>
<div>
Blindly grepping for these in network-manager source tree I found following:</div>
<div>
<blockquote class="tr_bq">
$ grep explicit_bzero -r configure.ac src/<br />configure.ac:<span style="white-space: pre;"> </span>explicit_bzero],<br />src/systemd/src/basic/string-util.h:void explicit_bzero(void *p, size_t l);<br />src/systemd/src/basic/string-util.c:void explicit_bzero(void *p, size_t l) {<br />src/systemd/src/basic/string-util.c: explicit_bzero(x, strlen(x));</blockquote>
</div>
<div>
First of all it seems like network-manager includes a partial embedded copy of systemd. Secondly that code is compiled into a temporary library and has autconf detection logic to use explicit_bzero. It also has an embedded implementation of explicit_bzero when it is not available in libc, however it does not have FORTIFY_SOURCES implementation of said function (__explicit_bzero_chk) as was later pointed out to me. And whilst this function is compiled into an intermediary noinst library, no functions that use explicit_bzero are used in the end by NetworkManger binary. To proof this, I've dropped <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=874585;filename=glibc-2.25-abi-drop.patch;msg=17" target="_blank">all code that uses explicit_bzero</a>, rebuild the package against glibc 2.26, and voila it only had Version reference on glibc 2.17 as expected from the end-result usage of shared symbols.</div>
<div>
<br /></div>
<div>
At this point toolchain bug was a suspect. It seems like whilst explicit_bzero shared symbol got optimised out, the version reference on 2.25 persisted to the linked binaries. At this point in the archive a snapshot version of binutils was in use. And in fact forcefully downgrading bintuils resulted in correct compilation / versions table referencing only glibc 2.17.</div>
<div>
<br /></div>
<div>
Mathias then took over a tarball of object files and filed upstream <a href="https://sourceware.org/bugzilla/show_bug.cgi?id=22150" target="_blank">bug report</a> against bintuils: "[2.29 Regression] ld.bfd keeps a version reference in .gnu.version_r for symbols which are optimized out". The discussion in that bug report is a bit beyond me as to me binutils is black magic. All I understood there was "we moved sweep and pass to another place due to some bugs", doing that introduced this bug, thus do multiple sweep and passes to make sure we fix old bugs and don't regress this either. Or something like that. Comments / Better description of the bintuils fix are welcomed.</div>
<div>
<br /></div>
<div>
Binutils got fixed by upstream developers, cherry-picked into debian, and ubuntu, network-manager got rebuild and everything is wonderful now. However, it does look like unused / deadend code paths tripped up optimisations in the toolchain which managed to slip by distribution package dependency generation and needless require a higher up version of glibc. I guess the lesson here is do not embed/compile unused code. Also I'm not sure why network-manager uses networkd internals like this, and maybe systemd should expose more APIs or serialise more state into /run, as most other things query things over dbus, private socket, or by establishing watches on /run/systemd/netif. I'll look into that another day.</div>
<div>
<br /></div>
<div>
Thanks a lot to Guillem Jover, Matthias Klose, Alan Modra, H.J. Lu, and others for getting involved. I would not be able to raise, debug, or fix this issue all by myself.</div>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0London, UK51.5073509 -0.1277582999999822351.1912379 -0.77320529999998222 51.8234639 0.51768870000001777tag:blogger.com,1999:blog-347582618045055410.post-70965599007475781642017-01-29T22:23:00.000+00:002017-01-29T22:23:57.238+00:002017 is the new 1984<div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><span style="margin-left: auto; margin-right: auto;"><a href="http://www.goodreads.com/book/show/5470.1984" target="_blank"><img border="0" src="http://t2.gstatic.com/images?q=tbn:ANd9GcQdCf2z_9xK2-HvOkf-wlKKgOW1m6-uBBBemJ1KDJO0NXb5nscf" height="400" width="245" /></a></span></td></tr>
<tr><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption"><span style="font-size: 12.8px;"><a href="http://www.goodreads.com/book/show/5470.1984" target="_blank">1984: Library Edition</a></span></td><td class="tr-caption"><span style="font-size: 12.8px;"><a href="http://www.goodreads.com/book/show/5470.1984" target="_blank">Novel by George Orwell, cover picture by Google Search result</a></span></td></tr>
</tbody></table>
I am scared.<br />
I am petrified.<br />
I am confused.<br />
I am sad.<br />
I am furious.<br />
I am angry.<br />
<br />
28 days later I shall return from NYC.<br />
<br />
I hope.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com1tag:blogger.com,1999:blog-347582618045055410.post-16692614858991839462016-12-21T18:24:00.001+00:002017-01-02T13:54:48.434+00:00Ubuntu Archive and CD/USB images complete migration to 4096 RSA signing keys<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://upload.wikimedia.org/wikipedia/commons/b/bd/Enigma_(crittografia)_-_Museo_scienza_e_tecnologia_Milano.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://upload.wikimedia.org/wikipedia/commons/b/bd/Enigma_(crittografia)_-_Museo_scienza_e_tecnologia_Milano.jpg" width="293" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Enigma machine photo by Alessandro Nassiri [<a href="http://creativecommons.org/licenses/by-sa/4.0">CC BY-SA 4.0</a>], <a href="https://commons.wikimedia.org/wiki/File%3AEnigma_(crittografia)_-_Museo_scienza_e_tecnologia_Milano.jpg">via Wikimedia Commons</a></td></tr>
</tbody></table>
<br />
Ubuntu Archive and CD/USB image use OpenPGP cryptography for verification and integrity protection. In 2012, a new archive signing key <a href="https://lists.ubuntu.com/archives/ubuntu-devel/2012-September/035903.html">was created</a> and we have started to dual-sign everything with both old and new keys.<br />
<br />
In April 2017, Ubuntu 12.04 LTS (Precise Pangolin) will go end of life. Precise was the last release that was signed with just the old signing key. Thus when Zesty Zapus is released as Ubuntu 17.04, there will no longer be any supported Ubuntu release that require the 2004 signing keys for validation.<br />
<br />
The Zesty Zapus release is now signed with just the 2012 signing key, which is 4096 RSA based key. The old 2004 signing keys, where were 1024 DSA based, have been removed from the default keyring and are no longer trusted by default in Zesty and up. The old keys are available in the removed keys keyring in the ubuntu-keyring package, for example in case one wants to verify things from <a href="http://old-releases.ubuntu.com/">old-releases.ubuntu.com</a>.<br />
<br />
Thus the signing key transition is coming to an end. Looking forward, I hope that by 18.04 LTS time-frame the <a href="https://en.wikipedia.org/wiki/SHA-3">SHA-3</a> algorithm will make its way into the OpenPGP spec and that we will possibly start a transition to 8096 RSA keys. But this is just wishful thinking as the current key strength, algorithm, and hashsums are deemed to be sufficient.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-24985836413828645082016-12-16T11:30:00.001+00:002016-12-16T11:30:34.811+00:00Swapfiles by default in Ubuntu<div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBC0TTWje3xZZQXj5RgHm1LHIG6y2T4-_wwyeZmHnC_QP2ikH54ORlOg8QG1rHEdN5zXLxyhT_xX-RGSKMAqWDSVCDHZ_c1dXKFj9PLV_dEpOAKF3Nghxu_miDup1tiiaBrTyAaxHnD_k/s1600/EmulexPersyst_4M_ISA.jpeg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBC0TTWje3xZZQXj5RgHm1LHIG6y2T4-_wwyeZmHnC_QP2ikH54ORlOg8QG1rHEdN5zXLxyhT_xX-RGSKMAqWDSVCDHZ_c1dXKFj9PLV_dEpOAKF3Nghxu_miDup1tiiaBrTyAaxHnD_k/s400/EmulexPersyst_4M_ISA.jpeg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">4MB RAM card</td></tr>
</tbody></table>
By default, in Ubuntu, we usually create a swap partition.<br />
<br />
Back in the day of 4MB RAM cards this made total sense, as the ration of RAM to disk space, was still very low. Things have changed since. Server, desktop, embedded systems have migrated to newer generations of both RAM and persistent storage. On the high performance side of things we see machines with faster storage in the form of NVMe and SSD drives. Reserving space for swap on such storage, can be seen as expensive and wasteful. This is also true for recent enough laptops and desktops too. Mobile phones have substantial amounts of RAM these days, and at times, coupled with eMMC storage - it is flash storage of lower performance, which have limited number of write cycles, hence should not be overused for volatile swap data. And there are also unicorns in a form of high performance computing of high memory (shared memory) systems with little or no disk space.<br />
<br />
Today, carving a partition and reserving twice the RAM size for swap makes little sense. For a common, general, machine most of the time this swap will not be used at all. Or if said swap space is in use but is of inappropriate size, changing it in-place in retrospect is painful.<br />
<br />
Starting from 17.04 Zesty Zapus release, instead of creating swap partitions, swapfiles will be used by default for non-lvm based installations.<br />
<br />
Secondly, the sizing of swapfiles is very different. It is no more than 5% of free disk space or 2GiB, whichever is lower.<br />
<br />
For preseeding, there are two toggles that control this behavior:<br />
<ul style="text-align: left;">
<li>d-i partman-swapfile/percentage string 5</li>
<li>d-i partman-swapfile/size string 2048</li>
</ul>
<div>
Setting either of those to zero, will result in system without any swap at all. And one can tweak relative integer percentage points and absolute limits in integer percentage points or MiB.</div>
<br />
On LVM based installations, swap logical volumes are used, since unfortunately LVM snapshots do not exclude swapfile changes. However, I would like to move partman-auto to respect the above proposed 5%-or-2GB limits.<br />
<br />
Ps. 4MB RAM card picture is by Bub's (Photo) [<a href="http://www.gnu.org/copyleft/fdl.html">GFDL</a> or <a href="http://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA-3.0</a>], <a href="https://commons.wikimedia.org/wiki/File%3AEmulexPersyst_4M_ISA.jpeg">via Wikimedia Commons</a></div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-3513662751322855192016-11-14T15:11:00.000+00:002016-11-14T15:11:12.670+00:00/boot less LVM rootfs in Zesty<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivTTqlmcmE3Tp89Yie19CSfVdjGlUvWue25jzSiQ1RQg_ceIjOuJnGHJjIbt1p7hFw7dzkTiOWFyqRmgH63qERNuZghDpO819G17J0AGmgKXAxTZN2N8966whL8yWIRyXE3IByJ-JlEAk/s1600/old-shoes-1466074526leD.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivTTqlmcmE3Tp89Yie19CSfVdjGlUvWue25jzSiQ1RQg_ceIjOuJnGHJjIbt1p7hFw7dzkTiOWFyqRmgH63qERNuZghDpO819G17J0AGmgKXAxTZN2N8966whL8yWIRyXE3IByJ-JlEAk/s400/old-shoes-1466074526leD.jpg" width="400" /></a></div>
<br />
On Ubuntu many of the default boot loaders support booting kernels located on LVM volumes. This includes following platforms<br />
<br />
<ul style="text-align: left;">
<li>i686, x86_64 bios grub2</li>
<li>arm64, armhf, i686, x86_64 UEFI grub2</li>
<li>PReP partitions on IBM PowerPC</li>
<li>zipl on IBM zSystems</li>
</ul>
<div>
For all of the above the d-i has been modified in Zesty to create LVM based installations without a dedicated /boot partition. We shall celebrate this achievement. Hopefully this means one doesn't need to remove kernels as much, or care about sizing /boot volume appropriately any more.</div>
<div>
<br /></div>
<div>
If there are more bootloaders in Ubuntu that support booting off LVM, please do get in touch with me. I'm interested if I can safely enable following platforms as well:</div>
<div>
<ul style="text-align: left;">
<li>armhf with u-boot</li>
<li>arm64 with u-boot</li>
<li>ppc64el with PReP volume</li>
</ul>
<div>
ps. boots pic is from <a href="http://www.publicdomainpictures.net/view-image.php?image=175609&picture=old-shoes">here</a></div>
</div>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-90723902291815777562016-06-25T20:24:00.001+01:002016-06-25T20:24:41.524+01:00Post-Brexit - The What Now?<div dir="ltr" style="text-align: left;" trbidi="on">
Out of 46,500,001 electorate 17,410,742 voted to leave, which is a mere 37.4% or just over a third. [<a href="http://www.electoralcommission.org.uk/find-information-by-subject/elections-and-referendums/upcoming-elections-and-referendums/eu-referendum/electorate-and-count-information">source</a>]. On my books this is not a clear expression of the UK wishes.<br />
<div>
<br /></div>
<div>
The reaction that the results have caused are devastating. The Scottish First Minister has announced plans for 2nd Scottish Independence referendum [<a href="http://www.snp.org/statement_on_euref_result_and_it_s_implications_for_scotland">source</a>]. Londoners are filing petitions calling for Independent London [<a href="https://www.change.org/p/sadiq-khan-declare-london-independent-from-the-uk-and-apply-to-join-the-eu">source</a>, <a href="https://petition.parliament.uk/petitions/133704/moderation-info">source</a>]. The Prime Minister announced his resignation [<a href="http://www.bbc.co.uk/news/uk-politics-36615028">source</a>]. Things are not stable.</div>
<div>
<br /></div>
<div>
I do not believe that super majority of the electorate are in favor of leaving the EU. I don't even believe that those who voted to leave have considered the break up of the UK as the inevitable outcome of the leave vote. There are numerous videos on the internet about that, impossible to quantify or reliably cite, but for example this [<a href="http://www.independent.co.uk/news/uk/politics/brexit-petition-latest-eu-referendum-rules-change-force-second-vote-poll-government-a7102486.html">source</a>]</div>
<div>
<br /></div>
<div style="text-align: center;">
<b>So What Now?</b></div>
<div style="text-align: center;">
<b><br /></b></div>
<div style="text-align: center;">
<b>P R O T E S T</b></div>
<div style="text-align: center;">
<b><br /></b></div>
<div style="text-align: left;">
I urge everyone to start protesting the outcome of the mistake that happened last Thursday. 4th of July is a good symbolic date to show your discontent with the UK governemnt and a tiny minority who are about to cause the country to fall apart with no other benefits. Please stand up and make yourself heard.</div>
<div>
<ul style="text-align: left;">
<li>Please <a href="http://www.bbc.co.uk/news/uk-politics-36615028">sign petition</a> for the 2nd EU Referendum</li>
</ul>
<ul style="text-align: left;">
<li>On Tuesday the 28th please come to <a href="https://www.facebook.com/events/1671704409745795/">Trafalgar Square to support London Stays campaign</a></li>
</ul>
<ul style="text-align: left;">
<li>General Strikes 4th & 5th of July</li>
</ul>
<ul style="text-align: left;"><ul>
<li>All 2 million EU citizens working in the UK Walk Out - <a href="https://www.facebook.com/events/278078699220301/?active_tab=posts">event</a></li>
<li>London National Black Bloc - <a href="https://www.facebook.com/events/1548749878752473/">GENERAL STRIKE SOLIDARITY LONDON</a> </li>
<li><a href="http://www.classwarparty.org.uk/event/toriesout-general-strike-protest-direct-action/">Glasgow Tories Out General Strike</a></li>
<li><a href="https://www.teachers.org.uk/campaigns/stand-up-for-education">National Union of Teachers - Stand Up for Eduction</a></li>
</ul>
</ul>
<div>
There are 64,100,000 people living in the UK according to the World Bank, maybe the government should fear and listen to the <a href="http://www.votenone.org.uk/uk-unheard-third.html">unheard third</a>. The current "majority" parliament was only elected by 24% of electorate.<br />
<br />
It is time for people to actually take control, we can fix our parliament, we can stop austerity, we can prevent the break up of the UK, and we can stay in the EU. Over to you.<br />
<br /></div>
</div>
<div style="text-align: center;">
<b>ps. How to elect next PM?</b><br />
<b><br /></b>
<div style="text-align: left;">
Electing next PM will be done within the Conservative Party, and that's kind of a bummer, given that the desperate state the country currently is in. It is not that hard to predict that Boris Johnson is a front-runner. If you wish to elect a different PM, I urge you to splash out 25 quid and register to be a member of the Conservative Party just for one year =) this way you will get a chance to directly elect the new Leader of the Conservative Party and thus the new Prime Minister. You can backdoor the Conservative election <a href="https://www.conservatives.com/join">here</a>.</div>
</div>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-65919044295643923662016-02-06T23:30:00.000+00:002016-02-06T23:30:08.356+00:00Blogging about Let's encrypt over HTTP<div dir="ltr" style="text-align: left;" trbidi="on">
So <a href="https://letsencrypt.org/">let's encrypt</a> thing started. And it can do challenges over http (serving text files) and over dns (serving .txt records).<br />
<br />
My "infrastructure" is fairly modest. I've seen too many of my email accounts getting swamped with spam, and or companies going bust. So I got my own domain name <a href="http://surgut.co.uk/">surgut.co.uk</a>. However, I don't have money or time to run my own services. So I've signed up for the Google Apps account for my domain to do email, blogging, etc.<br />
<br />
Then later i got the <a href="http://libnih.la/">libnih.la</a> domain to host API docs for the mentioned library. In the world of .io startups, I thought it's an incredibly funny domain name.<br />
<br />
But I also have a VPS to host static files on ad-hoc basis, run VPN, and an irc bouncer. My irc bouncer is ZNC and I used a self-signed certificate there, thus i had "ignore" ssl errors in all of my irc clients... which kind of defeats the purposes somewhat.<br />
<br />
I run my VPS on i386 (to save on memory usage) and on Ubuntu 14.04 LTS managed with Landscape. And my little services are just configured by hand there (not using juju).<br />
<br />
My first attempt at getting on the let's encrypt bandwagon was to use the official client. By fetching debs from xenial, and installing that on LTS. But the package/script there is huge, has support for things I don't need, and wants dependencies I don't have on 14.04 LTS.<br />
<br />
However I found a minimalist implementation <a href="http://letsencrypt.sh/">letsencrypt.sh</a> implemented in shell, with openssl and curl. It was trivial to get dependencies for and configure. Specified a domains text file, and that was it. And well, added sym links in my NGINX config to serve the challenges directory & a hook to deploy certificate to znc and restart that. I've added a cronjob to renew the certs too. Thinking about it, it's not complete as I'm not sure if NGINX will pick up certificate change and/or if it will need to be reloaded. I shall test that, once my cert expires.<br />
<br />
Tweaking config for NGINX was easy. And I was like, let's see how good it is. I pointed <a href="https://www.ssllabs.com/ssltest/">https://www.ssllabs.com/ssltest/</a> at my <a href="https://x4d.surgut.co.uk/">https://x4d.surgut.co.uk/</a> and I got a "C" rating. No forward secrecy, vulnerable to down grade attacks, BEAST, POODLE and stuff like that. I went googling for all types of NGINX configs and eventually found website with "best known practices" <a href="https://cipherli.st/">https://cipherli.st/</a> However, even that only got me to "B" rating, as it still has Diffie-Hellman things that ssltest caps at "B" rating. So I disabled those too. I've ended up with this gibberish:<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_protocols TLSv1 TLSv1.1 TLSv1.2;<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_prefer_server_ciphers on;<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_ciphers "EECDH+AESGCM:AES256+EECDH";<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_ecdh_curve secp384r1; # Requires nginx >= 1.1.0<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_session_cache shared:SSL:10m;<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>#ssl_session_tickets off; # Requires nginx >= 1.5.9<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_stapling on; # Requires nginx >= 1.3.7<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>ssl_stapling_verify on; # Requires nginx => 1.3.7<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>#resolver $DNS-IP-1 $DNS-IP-2 valid=300s;<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>#resolver_timeout 5s;<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>add_header X-Frame-Options DENY;<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>add_header X-Content-Type-Options nosniff;<br />
<div>
<br /></div>
<div>
I call it gibberish, because IMHO, I shouldn't need to specify any of the above... Anyway I got my A+ rating.</div>
<div>
<br /></div>
<div>
However, security is as best as the weakest link. I'm still serving things over HTTP, maybe I should disable that. And I'm yet to check how "good" the TLS is on my znc. Or if I need to further harden my sshd configuration.</div>
<div>
<br /></div>
<div>
This has filled a big gap in my infrastructure. However a few things remain served over HTTP only.</div>
<div>
<br /></div>
<div>
<a href="http://blog.surgut.co.uk/">http://blog.surgut.co.uk</a> is hosted by an Alphabet's / Google's Blogger service. Which I would want to be served over HTTPS.</div>
<div>
<br /></div>
<div>
<a href="http://libnih.la/">http://libnih.la</a> is hosted by GitHub Inc service. Which I would want to be served over HTTPS.</div>
<div>
<br /></div>
<div>
I do not want to manage those services, experience load / spammers / DDoS attacks etc. But I am happy to sign CSRs with let's encrypt and deploy certs over to those companies. Or allow them to self-obtain certificates from let's encrypt on my behalf. I used <a href="http://gandi.net/">gandi.net</a> as my domain names provider, which offers an RPC API to manage domains and their zones files, thus e.g. I can also generate an API token for those companies to respond with a dns-01 challenge from let's encrypt.</div>
<div>
<br /></div>
<div>
One step at a time I guess.</div>
<br />
The postings on this site are my own and don't necessarily represent any past/present/future employers' positions, strategies, or opinions.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com1tag:blogger.com,1999:blog-347582618045055410.post-1320674489117277212016-01-30T01:37:00.001+00:002016-01-30T01:39:43.523+00:00Four gunmen outside<p dir="ltr">There are four gunmen outside of my hotel. They are armed with automatic rifles and pistols. I am scared for my life having sneaked past them inside. Everyone else is acting as if everything is normal. Nobody is scared or running for cover. Nobody called the police. I've asked the reception to talk to the gunmen and ask them to leave. They looked at me as if I am mad. Maybe I am. Is this what shizophrenia feels like?! Can you see them on the picture?! Please help. There are four gunmen outside of my hotel. I am not in central Beirut, I am in central Brussels.</p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi48fJridajbDjER5RSoRsQPslhV4UTw7BQ_AA5PiwKPK4xBCiNtvp-z2IlOcWbQF6m7B0ags4fFmwNFq3xz_dXKH3WYM-rRqx-x29n5I3sdyZyAc3SHBn4pDUlrmC_n-2IisnLt6l-ozc/s1600/IMG_20160129_193735.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"> <img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi48fJridajbDjER5RSoRsQPslhV4UTw7BQ_AA5PiwKPK4xBCiNtvp-z2IlOcWbQF6m7B0ags4fFmwNFq3xz_dXKH3WYM-rRqx-x29n5I3sdyZyAc3SHBn4pDUlrmC_n-2IisnLt6l-ozc/s640/IMG_20160129_193735.jpg"> </a> </div>Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0Brussels, Brussels50.850338 4.3517103tag:blogger.com,1999:blog-347582618045055410.post-43454533707606934302015-10-12T11:23:00.000+01:002015-10-12T11:23:06.055+01:00uwsgi gains --paste-name option<div dir="ltr" style="text-align: left;" trbidi="on">
One of the <a href="https://www.python.org/dev/peps/pep-3333/">WSGI</a> servers that is commonly using with <a href="http://nginx.org/">nginx</a> is <a href="https://uwsgi-docs.readthedocs.org/en/latest/">uwsgi</a>. One way to configure and deploy simple and complex WSGI middlewares and apps is by using <a href="http://pythonpaste.org/deploy/">Paste deploy</a> tooling. However, until now uwsgi was only able to load a default app under the suffix "main". There is no such limitation in the Paste deploy code itself, as functions there accept a name argument. What was missing is an option in uwsgi to pass, optionally, alternative non-default name of the app to load.<br />
<br />
A <a href="https://github.com/clearlinux/uwsgi/commit/a6005624af58c44114635489b5bdd420884aa149">patch</a> adding this option has now been <a href="https://github.com/unbit/uwsgi/pull/1066#event-432027974">merged upstream</a> and should be available in the next uwsgi release.<br />
<br />
What I was hoping that one will be able to do something like this:<br />
<blockquote class="tr_bq">
$ uwsgi --ini-paste /etc/nova/api-paste.ini --paste-name osapi_compute</blockquote>
But alas that didn't work. I wish all OpenStack Software service were deployable as normal wsgi stand-alone apps without any addition glue code. Keystone is kind of like this with /usr/share/httpd/cgi-bin/keystone/main entry point for deploying keystone as a wsgi app, instead of a daemon.<br />
<br />
Also can nova operate on top of nginx ugreen threats? Or is that something entirely difference from the recent day eventlet?!<br />
<br />
The postings on this site are my own and don't necessarily represent Intel’s positions, strategies, or opinions.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-37117701397596310422015-09-18T22:11:00.000+01:002015-09-18T22:11:09.170+01:00Clear Containers for Docker* Engine<div dir="ltr" style="text-align: left;" trbidi="on">
Today at work, I announced something James Hunt, Ikey Doherty and myself have been working on. We integrated <a href="https://lwn.net/Articles/644675/">Clear Containers</a> technology with <a href="https://www.docker.com/">Docker* Engine</a> to create <a href="https://lists.clearlinux.org/pipermail/dev/2015-September/000049.html">Clear Containers for Docker* Engine</a>.<br />
<br />
After following <a href="https://software.opensuse.org/download.html?project=home%3Aclearlinux%3Apreview&package=clear-containers-docker">installation instructions</a>, one can pull and run existing Docker* containers in the secure Clear Containers environment. This means that instead of namespaces, a fast virtual machine is started using the kvmtool hypervisor. This VM is running an optimised minimal Linux* kernel and the optimised Clear Linux* for Intel<span style="white-space: pre-wrap;">®</span> Architecture Project user-space, with the only goal to execute the Docker* workload and then shut down.<br />
<br />
The net effect is almost indistinguishable from typical Docker* container usage:<br />
<blockquote class="tr_bq">
$ docker run -ti ubuntu:vivid<br />root@d88a60502ed7:/# systemd-detect-virt<br />kvm</blockquote>
Apart from, as you see, it's running inside a kvm VM, and thus protected by <span style="white-space: pre-wrap;">Intel® Virtualization Technology.</span><br />
<span style="white-space: pre-wrap;"><br /></span>
<span style="white-space: pre-wrap;">This is available on Clear Linux* as well as multiple other operating systems.</span><br />
<br />
I hope this is exciting enough for people to try out, and if you have any feedback, feel free to leave comments or join our <a href="https://lists.clearlinux.org/mailman/listinfo/dev">mailing list</a>.<br /><br />*Other names and brands may be claimed as the property of others<br /><br />The postings on this site are my own and don't necessarily represent Intel’s positions, strategies, or opinions.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-35760281787363567562015-08-27T11:39:00.000+01:002015-08-28T10:48:57.355+01:00Go enjoy Python3<div dir="ltr" style="text-align: left;" trbidi="on">
Given a string, get a truncated string of length up to 12.<br />
<br />
The task is ambiguous, as it doesn't say anything about whether or not 12 should include terminating null character or not. None the less, let's see how one would achieve this in various languages.<br />
Let's start with python3
<br />
<blockquote class="tr_bq">
<pre><code>
import sys
print(sys.argv[1][:12])</code>
</pre>
</blockquote>
Simple enough, in essence given first argument, print it up to length 12. As an added this also deals with unicode correctly that is if passed arg is 車賈滑豈更串句龜龜契金喇車賈滑豈更串句龜龜契金喇, it will correctly print 車賈滑豈更串句龜龜契金喇. (note these are just random Unicode strings to me, no idea what they stand for).
<br />
<br />
In C things are slightly more verbose, but in essence, I am going to use strncpy function:
<br />
<blockquote class="tr_bq">
<pre><code>
#include <stdio.h>
#include <string.h>
void main(int argc, char *argv[]) {
char res[12];
strncpy(res,argv[1],12);
printf("%s\n",res);
}
</code></pre>
</blockquote>
This treats things as byte-array instead of unicode, thus for unicode test it will end up printing just 車賈滑豈. But it is still simple enough.
<br />
Finally we have Go
<br />
<blockquote class="tr_bq">
<pre><code>package main
import "os"
import "fmt"
import "math"
func main() {
fmt.Printf("%s\n", os.Args[1][:int(math.Min(12, float64(len(os.Args[1]))))])
}
</code></pre>
</blockquote>
This similarly treats argument as a byte array, and one needs to cast the argument to a rune to get unicode string handling. But there are quite a few caveats. One cannot take out of bounds slices. Thus a naïve os.Args[1][:12] can result in a runtime panic that slice bounds are out of range. Or if a string is known at compile time, a compile time error. Hence one needs to calculate length, and do a min comparison. And there lies the next caveat, math.Min() is only defined for float64 type, and slice indexes can only be integers and thus we end up writing ]))))])...<br />
<br />
12 points for python3, 8 points for C, and Go receives nul points Eurovision style.<br />
<br />
<b>EDIT: </b><span class="Ub gna" style="-webkit-transition: color 0.218s initial initial; background-color: white; color: #262626; cursor: pointer; font-family: Roboto, arial, sans-serif; font-size: 13px; font-weight: bold; line-height: 18.2000007629395px; text-decoration: none; transition: color 0.218s initial initial;"><a href="https://apis.google.com/u/0/wm/1/101919461013229598346" oid="101919461013229598346" style="background-color: white; color: #427fed; cursor: pointer; font-family: Roboto, arial, sans-serif; font-size: 13px; line-height: 18.2000007629395px; text-decoration: none;" target="_blank">Andreas Røssland</a> and James Hunt are full of win. Both suggesting fmt.Printf("%.12s\n", os.Args[1]) for go. I like that a lot, as it gives simplicity & readability without compromising the default safety against out of bounds access. Hence the scores are now: 14 points for Go, 12 points for python3 and 8 points for C.</span><br />
<br />
<b>EDIT2:</b> I was pointed out much better C implementation by Keith Thompson - <a href="http://pastebin.com/5i7rFmMQ">http://pastebin.com/5i7rFmMQ</a> in essence it uses strncat() which has much better null termination semantics. And Ben posted a C implementation which handles wide characters <a href="http://www.decadent.org.uk/ben/blog/truncating-a-string-in-c.html">http://www.decadent.org.uk/ben/blog/truncating-a-string-in-c.html</a>. I regret to inform you that this blog post got syndicated onto hacker news and has now become the top viewed post on my blog of all time, overnight. In retrospect, I regret awarding points at the end of the blog post, as that's just was merely an expression of opinion and is highly subjective measure. But this problem statement did originate from me reviewing go code that did "if/then/else" comparison and got it wrong to truncate a string and I thought surely one can just do [:12] which has lead me down the rabbit hole of discovering a lot about Go; it's compile and runtime out of bounds access safeguards; lack of universal Min() function; runes vs strings handling and so on. I'm only a beginner go programmer and I am very sorry for wasting everyone's time on this. I guess people didn't have much to do on a Throwback Thursday.<br />
<br />
The postings on this site are my own and don't necessarily represent Intel’s positions, strategies, or opinions.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com1tag:blogger.com,1999:blog-347582618045055410.post-46933298710450354772015-03-30T16:15:00.001+01:002015-03-30T16:15:45.245+01:00Boiling frog, or when did we loose it with /etc ?<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<div style="text-align: left;">
<blockquote class="tr_bq" style="text-align: left;">
$ sudo find /etc -type f | wc -l<br />
2794</blockquote>
</div>
</blockquote>
</blockquote>
<h4 style="text-align: left;">
Stateless</h4>
When was the last time you looked at /etc and thought - "I honestly know what every single file in here is". Or for example had a thought "Each file in here is configuration changes that I made". Or for example do you have confidence that your system will continue to function correctly if any of those files and directories are removed?<br />
<br />
Traditionally most *NIX utilities are simple enough utilities, that do not require any configuration files what's so ever. However most have command line arguments, and environment variables to manipulate their behavior. Some of the more complex utilities have configuration files under /etc, sometimes with "layer" configuration from user's home directory (~/). Most of them are generally widely accepted. However, these do not segregate upstream / distribution / site administrator / local administrator / user configuration changes. Most update mechanisms created various ways to deal with merging and maintaining the correct state of those. For example both <a href="https://www.debian.org/doc/debian-policy/ch-files.html#s-config-files">dpkg</a> & <a href="http://www.rpm.org/max-rpm/s1-rpm-inside-files-list-directives.html">RPM (%config)</a> have elaborate strategies and policies and ways to deal with them. However, even today, still, they cause problems: prompting user for whitespace changes in config files, not preserving user changes, or failing to migrate them.<br />
<br />
I can't find exact date, but it has now been something like 12 years since <a href="http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html">XDG Base directory specification</a> was drafted. It came from Desktop Environment requirements, but one thing it achieves is segregation between upstream / distro / admin / user induced changes. When applications started to implement Base directory specification, I started to feel empowered. Upstream ships sensible configs in /usr, distribution integrators ship their overlay tweaks packaged in /usr, my site admin applies further requirements in /etc, and as I user I am free to improve or brake everything with configs in ~/. One of the best things from this setup - no upgrade prompts, and ease of reverting each layer of those configs (or at least auditing where the settings are coming from).<br />
<br />
However, the uptake of XDG Base directory spec is slow / non-existing among the core components of any OS today. And at the same time /etc has grown to be a dumping ground for pretty much everything under the sun:<br />
<ul style="text-align: left;">
<li>Symlink farms - E.g. /etc/rc*.d/*, /etc/systemd/system/*.wants/*, /etc/ssl/certs/*</li>
</ul>
<ul style="text-align: left;">
<li>Cache files - E.g. /etc/ld.so.cache</li>
</ul>
<ul style="text-align: left;">
<li>Empty (and mandatory) directories</li>
</ul>
<ul style="text-align: left;">
<li>Empty (and mandatory) "configuration" files. - E.g. whitespace & comments only</li>
</ul>
Let's be brutally honest and say that none of the above belongs in /etc. /etc must be for end-user configuration only, made by the end user alone and nobody else (or e.g. an automation tool driven by the end-user, like puppet).<br />
<br />
Documentation of available configuration options and syntax to specify those in the config files should be shipped... in the documentation. E.g. man pages, /usr/share/doc, and so on. And not as the system-wide "example" config files. Absence of the files in /etc must not be treated as fatal, but a norm, since most users use default settings (especially for the most obscure options). Lastly compiled-in defaults should be used where possible, or e.g. layer configuration from multiple locations (e.g. /usr, /etc, ~/ where appropriate).<br />
<br />
Above observations are not novel, and shared by most developers and users in the wider open source ecosystem. There are many projects and concepts to deal with this problem by using automation (e.g. puppet, chef), by migrating to new layouts (e.g. implementing / supporting XDG base dir spec), using "app bundles" (e.g. mobile apps, docker), or fully enumerating/abstracting everything in a generic manner (e.g. <a href="http://nixos.org/">NixOS</a>). Whilst fixing the issue at hand, these solutions do increase the dependency on files in /etc to be available. In other words we grew a de-facto user-space API we must not break, because modifications to the well known files in /etc are expected to take effect by both users and many administrator tools.<br />
<br />
Since August last year, I have joined <a href="https://01.org/">Open Source Technology Center</a> at <a href="http://www.intel.com/">Intel</a>, and have been working on <a href="https://clearlinux.org/">Clear Linux* Project for Intel Architecture</a>. One of the goals we have set out is to achieve stateless operation - that is to have empty /etc by default, reserved for user modification alone, yet continuing to support all legacy / well-known configuration paths. The premise is that all software can be patched with auto-detection, built-in defaults or support for layered configuration to achieve this. I hope that this work would interest everyone and will be widely adopted.<br />
<br />
Whilst the effort to convert everything is still on going, I want to discuss a few examples of any core system.<br />
<h4 style="text-align: left;">
Shadow</h4>
The <a href="http://linux.die.net/man/1/login">login(1)</a> command, whilst having built-in default for every single option exits with status 1, if it cannot <a href="http://linux.die.net/man/2/stat">stat(2)</a> <a href="http://linux.die.net/man/5/login.defs">login.defs(5)</a> file.<br />
<br />
The <a href="http://linux.die.net/man/1/passwd">passwd(1)</a> command will write out the salted/hashed password in the <a href="http://linux.die.net/man/5/passwd">passwd(5)</a> file, rather than in <a href="http://linux.die.net/man/5/shadow">shadow(5)</a>, if it cannot stat the shadow(5) file. There is similar behavior with gshadow. I found it very ironic, that upstream project "shadow" does not use shadow(5) by default.<br />
<br />
Similarly, stock files manipulated by passwd/useradd/groupadd utilities are not created, if missing.<br />
<br />
Some settings in login.defs(5) are not applicable, when compiled with PAM support, yet present in the default shipped login.defs(5) file.<br />
<br />
Patches to resolve above issues are undergoing review on the <a href="http://lists.alioth.debian.org/pipermail/pkg-shadow-devel/2015-March/010589.html">upstream mailing list</a>.<br />
<h4>
DBus</h4>
In xml based configuration, `includedir' elements are mandatory to exist on disk, that is empty directory must be present, if referenced. If these directories are non-existant, the configuration fails to load and the system or session bus are not started.<br />
<br />
Similarly, upstream have general agreement with the stateless concept and patches to move all of dbus default configurations from /etc to /usr are being reviewed for inclusion at <a href="https://bugs.freedesktop.org/show_bug.cgi?id=89280">the bug tracker</a>. I hope this change will make into the 1.10 stable release.<br />
<h4 style="text-align: left;">
GNU Lib C</h4>
Today, we live in a dual-stack IPv4 and IPv6 world, where even the localhost has multiple IP addresses. As a slightly ageist time reference, the first VCS I ever used was git. Thus when I read below, I get very confused:<br />
<blockquote class="tr_bq">
$ cat /etc/host.conf<br />
# The "order" line is only used by old versions of the C library.<br />
order hosts,bind<br />
multi on</blockquote>
Why not simply do this:<br />
<blockquote class="tr_bq">
--- a/resolv/res_hconf.c<br />
+++ b/resolv/res_hconf.c<br />
@@ -309,6 +309,8 @@ do_init (void)<br />
if (hconf_name == NULL)<br />
hconf_name = _PATH_HOSTCONF;<br />
<br />
+ arg_bool (ENV_MULTI, 1, "on", HCONF_FLAG_MULTI);<br />
+<br />
fp = fopen (hconf_name, "rce");<br />
if (fp)<br />
{</blockquote>
<div>
<br /></div>
There are still many other packages that needed fixes similar to above. Stay tuned for further stateless observations about Glibc, OpenSSH, systemd and other well known packages.<br />
<br />
In the mean time, you can try out <a href="https://clearlinux.org/">https://clearlinux.org/</a> images that implement above and more already. If you want to chat about it more, comment on G+, find myself on irc - xnox @ <a href="irc://irc.freenode.net/clearlinux">irc.freenode.net</a> #clearlinux and join our <a href="https://lists.clearlinux.org/mailman/listinfo/dev">mailing list</a> to kick the conversation off, if you are interested in making the world more stateless.<br />
<br />
ps.<br />
I am a professional Linux Distribution developer, currently employed by Intel, however the postings on this site are my own and don't necessarily represent Intel's or any other past/present/future employer positions, strategies, or opinions.<br />
<br />
* Other names and brands may be claimed as the property of others<br />
<div>
<br /></div>
<br /></div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0London, UK51.5073509 -0.1277582999999822351.1912379 -0.77320529999998222 51.8234639 0.51768870000001777tag:blogger.com,1999:blog-347582618045055410.post-19163482213339940932015-03-15T23:30:00.000+00:002015-03-15T23:30:05.524+00:00My IDE needs a makeover<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">
Current Setup</h2>
I am a Linux Distribution Engineer and work on arbitrary open source projects. Mostly I'm patching/packaging existing things, and sometimes start fresh projects.<br />
<br />
My "IDE", or rather I shall say "toolbox" is rather sparse:<br />
<br />
<ul style="text-align: left;">
<li>GNOME Terminal</li>
<li>Google Chrome</li>
<li>GNU Emacs</li>
<li>GCC toolcahin with GDB</li>
<li>Python3 - iPython, iPdb, pyflakes</li>
<li>git, GNU bazaar</li>
</ul>
<div>
There are a few things that annoy me, and should be done better these days.</div>
<h2 style="text-align: left;">
Documentation</h2>
<div>
I lookup documentation mostly with Google Chrome. This includes the texinfo renderings of the docs. There are a few reasons for that. First of all my developer machine is not polluted with all the dev packages under the sun, instead I compile practically everything in a chroot. And most of the time chroots have much newer versions of everything (from gcc & automake, to boost and whatever other dependencies are in use). However I would like to have easy generic lookup builtin for common things that I lookup in the references and which have not changed for a long time:</div>
<div>
<ul style="text-align: left;">
<li>gcc builtins & defines</li>
<li>glibc functions</li>
<li>automake/autoconf functions definitions</li>
</ul>
<div>
Given that my preferred editor is Emacs, it should be natural to use `info' mode to look things up. However, the rendering there is archaic and is really hard to read. At least when visiting the HTML renderings, the function names are in <b>bold</b> and stand out from the rest of the description.</div>
</div>
<div>
<br /></div>
<div>
Ideally I would have unified place to lookup docs, instead of using Google Chrome and navigating: gnu.org, gnome.org, readthedocs.org, freedesktop.org.</div>
<h2 style="text-align: left;">
Project Management</h2>
<div>
I really hate "traditional" IDEs that create and pollute the working directories with random extra files. My project management tool is VCS, thus .git should be automatically recognized as a "project". I should be able to navigate repository files, have them scanned for tab-completion and jumping to symbols and the like. At the moment, I exit the editor and use git grep to find things and open those files in the editor again. I don't use any tagging systems at the moment, ideally git repository would be scanned and Exuberant Tags (this seems to be the latest hotness in tagging space) stored inside the .git directory automatically.</div>
<h2 style="text-align: left;">
"SDK" aware aka chroot support</h2>
<div>
The IDE should be aware of chroots, how to compile things in a chroot and ideally how to compile packages with sbuild, mock or obs build (these are apt, yum and zypper preferred solutions for package compilation). Most importantly to use those chroots to tag includes headers for tab completion.</div>
<h2 style="text-align: left;">
Shell</h2>
<div>
Gnome Terminal is good enough for my needs. I do have a problem of too many terminal windows... I have tried Terminator (a tiling single-window / multiple-tabs terminal). However during development the things I use shell for, should be part of the IDE directly: changing projects, opening/closing/navigating/creating files, invoking build, invoking debug, "refactoring" (sed). I think I do want to try out a pull-down terminal for temporal look-ups together with a tiling "main" terminal. Or ideally ditch it all together. Emacs does provide multiple terminals, but when I did that I ended up with "inception" -> launching an instance of emacs, inside the terminal, inside emacs...</div>
<h2 style="text-align: left;">
Conclusion</h2>
<div>
If anybody has tips or suggestions do share. I will investigate and experiment with all of the above, and see if I can experiment and find new cool things that work better than my current setup.</div>
<div>
<br /></div>
<div>
<br /></div>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-5303424764661250892015-03-14T15:19:00.001+00:002015-03-14T15:19:36.446+00:00Intel CPU microcode support in ubuntu-drivers-common<div dir="ltr" style="text-align: left;" trbidi="on">
Ubuntu Vivid Vervet 15.04 is on its final approach to release at the end of next month. Here is a highlight of one of the features that I have helped to land.<br />
<br />
ubuntu-drivers-common is a framework to detect hardware-dependent components on user's machine and offer to install additional packages to enable better support for such hardware. Typical examples are drivers for the graphics cards. This cycle I have added CPU family detection plugin, which helps to detect cpu family and install appropriate microcode update. E.g. if one is running Intel CPU, intel-microcode package is installed.<br />
<br />
Check out:<br />
<blockquote class="tr_bq">
$ ubuntu-drivers devices<br />
$ ubuntu-drivers list<br />
$ ubuntu-drivers autoinstall</blockquote>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-14234406524049812252015-01-21T00:06:00.000+00:002015-01-21T00:06:12.155+00:00Python 3 ports of launchpadlib & ubuntu-dev-tools (library) are available<div dir="ltr" style="text-align: left;" trbidi="on">
I'm happy to announce that Python 3 ports of launchpadlib & ubuntu-dev-tools (library) are available for consumption.<br />
<br />
These are 1.10.3 & 0.155 respectfully.<br />
<br />
This means that everyone should start porting their reports, tools, and scriptage to python3.<br />
<br />
ubuntu-dev-tools has the library portion ported to python3, as I did not dare to switch individual scripts to python3 without thorough interactive testing. Please help out porting those and/or file bug reports against the python3 port. Feel free to subscribe me to the bug reports on launchpad.<br />
<br />
For the time being, I believe some things will not be easy to port to python3 because of the elephant in the room - bzrlib. For some things like lp-shell, it should be easy to move away from bzrlib, as non-vcs things are used there. For other things the current suggestion is to probably fork to bzr binary or a python2 process. I ponder if a minimal usable python3-bzrlib wrapper around python2 bzrlib is possible to satisfy the needs of basic and common scripts.<br />
<br />
On a side note, launchpadlib & lazr.restfulclient have out of the box proxy support enabled. This makes things like add-apt-repository work behind networks with such setup. I think a few people will be happy about that.<br />
<br />
All of these goodies are available in Ubuntu 15.04 (Vivid Vervet) or Debian Experimental (and/or NEW queue).</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-8913169667444946802014-11-23T21:15:00.001+00:002014-11-23T21:15:51.202+00:00Analyzing public OpenPGP keys<div dir="ltr" style="text-align: left;" trbidi="on">
OpenPGP Message Format (<a href="https://tools.ietf.org/html/rfc4880" target="_blank">RFC 4880</a>) well defines key structure and wire formats (openpgp packets). Thus when I looked for public key network (SKS) server setup, I quickly found pointers to dump files in said format for bootstrapping a key server.<br />
<br />
I did not feel like experimenting with Python and instead opted for Go and found <a href="http://code.google.com/p/go.crypto/openpgp/packet">http://code.google.com/p/go.crypto/openpgp/packet</a> library that has comprehensive support for parsing openpgp low level structures. I've downloaded the SKS dump, verified it's MD5SUM hashes (lolz), and went ahead to process them in Go.<br />
<br />
With help from <a href="http://github.com/lib/pq">http://github.com/lib/pq</a> and database/sql, I've written a small program to churn through all the dump files, filter for primary RSA keys (not subkeys) and inject them into a database table. The things that I have chosen to inject are fingerprint, N, E. N & E are the modulus of the RSA key pair and the public exponent. Together they form a public part of an RSA keypair. So far, nothing fancy.<br />
<br />
Next I've run an SQL query to see how unique things are... and found 92 unique N & E pairs that have from two and up to fifteen duplicates. In total it is 231 unique fingerprints, which use key material with a known duplicate in the public key network. That didn't sound good. And also odd - given that over 940 000 other RSA keys managed to get unique enough entropy to pull out a unique key out of the keyspace haystack (which is humongously huge by the way).<br />
<br />
Having the list of the keys, I've fetched them and they do not look like regular keys - their UIDs do not have names & emails, instead they look like something from the <a href="http://web.monkeysphere.info/" target="_blank">monkeysphere</a>. The keys look like they are originally used for TLS and/or SSH authentication, but were converted into OpenPGP format and uploaded into the public key server. This reminded me of the Debian's SSL key generation vulnerability <a href="https://wiki.debian.org/SSLkeys" target="_blank">CVE-2008-0166</a>. So these keys might have been generated with bad entropy due to affected tools by that CVE and later converted to OpenPGP.<br />
<br />
Looking at the <a href="https://tracker.debian.org/pkg/openssl-blacklist">openssl-blacklist</a> package, it should be relatively easy for me to generate all possible RSA key-pairs and I believe all other material that is hashed to generate the fingerprint are also available (<a href="https://tools.ietf.org/html/rfc4880#section-12.2">RFC 4880#12.2</a>). Thus it should be reasonably possible to generate matching private keys, generate revocation certificates and publish the revocation certificate with pointers to CVE-2008-0166. (Or email it to the people who have signed given monkeysphered keys). When I have a minute I will work on generating openpgp-blacklist type of scripts to address this.<br />
<br />
If anyone is interested in the Go source code I've written to process openpgp packets, please drop me a line and I'll publish it on github or something.</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-65254786358780905932014-08-03T05:21:00.000+01:002014-08-03T05:30:09.576+01:00What is net neutrality?<div dir="ltr" style="text-align: left;" trbidi="on">
<center>
<div dir="ltr" style="background-color: #F1F1F1; border-color: black; border-radius: 30px; border: 2px solid; padding: 10px; text-align: left; width: 400px;" trbidi="on">
<h1 style="font-family: Arial, sans-serif; font-size: 20px; line-height: 1.2em; font-weight: bold;">
Sorry, the web page you have requested is not available through your internet connection.</h1>
<h1 style="text-align: center;">
<div style="font-family: Arial, sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; margin-bottom: 18.08px; text-align: left;">
We have received an order from the Courts requiring us to prevent access to this site in order to help protect against Lex Julia Majestatis infridgement.</div>
<hr style="font-family: Arial, sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; text-align: left;" />
<div style="font-family: Arial, sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; margin-top: 18.08px; text-align: left;margin-top: 18px;">
If you are a home broadband customer, for more information on why certain web pages are blocked, please click <a href="https://www.eff.org/deeplinks/content-blocking" style="color: #cc0000; text-decoration: none;" target="_blank" title="Home broadband">here</a>.</div>
<div style="font-family: Arial, sans-serif; font-size: 16px; font-weight: normal; line-height: 18.079999923706055px; text-align: left; margin-top: 18px;">
If you are a business customer, or are trying to view this page through your company's internet connection, please click <a href="https://www.eff.org/deeplinks/content-blocking" style="color: #cc0000; text-decoration: none;" target="_blank" title="Business">here</a>.
<div style="font-family: Arial, sans-serif; font-size: 80px; font-weight: normal; line-height: 18.079999923706055px; margin-bottom: 18.08px; text-align: left; color: red;margin-top: 18px;">
∞
</div>
</div>
</h1>
</div>
</center>
</div>Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-71386508719187070102014-07-04T18:54:00.000+01:002014-07-13T13:32:12.084+01:00Hacking on launchpadlib<div dir="ltr" style="text-align: left;" trbidi="on">
So here is a quick sample of my progress playing around with launchpadlib using lp-shell from lptools:
<br />
<pre style="white-space: pre-wrap; word-wrap: break-word;">In [1]: lp
Out[1]: <launchpadlib.launchpad.Launchpad at 0x7f49ecc649b0>
In [2]: lp.distributions
Out[2]: <launchpadlib.launchpad.DistributionSet at 0x7f49ddf0e630>
In [3]: lp.distributions['ubuntu']
Out[3]: <distribution at https://api.launchpad.net/1.0/ubuntu>
In [4]: lp.distributions['ubuntu'].display_name
Out[4]: 'Ubuntu'
In [5]: lp.distributions['ubuntu'].summary
Out[5]: 'Ubuntu is a complete Linux-based operating system, freely available with both community and professional support.'
In [7]: import sys; print(sys.version)
3.4.1 (default, Jun 9 2014, 17:34:49)
[GCC 4.8.3]</pre>
<br />
There is not much yet, but it's a start. python3 port of launchpadlib is coming soon. It has been attempted a few times before and I am leveraging that work. Porting this stack has proven to be the most difficult python3 port I have ever done. But there is always python-libvirt that still needs porting ;-)<br />
<br />
Some of above is just merge proposals against launchpadlib & lazr.restfulclient, and requires not yet packaged modules in the archive. When trying it out, I'm still getting a lot of run-time asserts and things that haven't been picked up by e.g. pyflakes3 and has not been unit-tested yet.
<script src="https://raw.github.com/moski/gist-Blogger/master/public/gistLoader.js" type="text/javascript"></script>
</div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0tag:blogger.com,1999:blog-347582618045055410.post-4465558472672652092014-06-10T19:59:00.000+01:002014-06-10T19:59:00.209+01:00cross-compile go code, including cgo<div dir="ltr" style="text-align: left;" trbidi="on">
By all means cross-compiling a new language/stack is not going to be pretty, but it didn't turn out that bad.<br />
<br />
A few weeks back, I was told that go code which uses cgo (that is utilising C api calls to shared libraries exporting C interface) cannot be cross-compiled. Well, if it's just calling out a C compiler it should totally be easy to cross compile, since so much of our platform is.<br />
<br />
So there we go, first I've picked a moderately small project which only does a couple cgo calls, and check that it compiles correctly:<br />
<br />
<blockquote class="tr_bq">
$ sudo apt-get build-dep ubuntu-push-client<br />$ go get launchpad.net/ubuntu-push/...<br />$ cd $GOPATH/src/launchpad.net/ubuntu-push/<br />$ go build ubuntu-push-client.go</blockquote>
Well, when your gcc is all is easy.<br />
<br />
I didn't want to polute my system, so I quickly created a chroot with go, build-dependencies in armhf architectures and cross-compiler:<br />
<br />
<blockquote class="tr_bq">
# Get a chroot with build-dependencies installed, I am basing on top of a click-chroot<br /># one should be able to use any chroot which is armhf multiarch enabled.<br />$ sudo click chroot -aarmhf -fubuntu-sdk-14.04 -s utopic create<br />$ sudo click chroot -aarmhf -fubuntu-sdk-14.04 -s utopic maint apt-get install golang-go golang-go-linux-arm golang-go-dbus-dev golang-go-xdg-dev golang-gocheck-dev golang-gosqlite-dev golang-uuid-dev libgcrypt11-dev:armhf libglib2.0-dev:armhf libwhoopsie-dev:armhf libubuntuoneauth-2.0-dev:armhf libdbus-1-dev:armhf libnih-dbus-dev:armhf libsqlite3-dev:armhf crossbuild-essential-armhf</blockquote>
After that the tricky bit was advising go to cross-compile:<br />
<br />
<blockquote class="tr_bq">
$ click chroot -aarmhf -fubuntu-sdk-14.04 -s utopic run CGO_ENABLED=1 GOARCH=arm GOARM=7 PKG_CONFIG_LIBDIR=/usr/lib/arm-linux-gnueabihf/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig GOPATH=/usr/share/gocode/:~/go CC=arm-linux-gnueabihf-gcc go build -ldflags '-extld=arm-linux-gnueabihf-gcc' ubuntu-push-client.go</blockquote>
<div>
Ignoring the click chroot wrapper:</div>
<div>
<ul style="text-align: left;">
<li>CGO_ENABLED=1 - by default cgo is disabled when cross-compiling, but really shouldn't be as compiler names are standard $(GNU_TRIPPLET) prefixed tools</li>
<li>GOARCH=arm - set the target arch</li>
<li>GOARM=7 - set ABI level</li>
<li>PKG_CONFIG_LIBDIR - the ugly beast to pass where pkg-config should search for .pc files. With autoconf one simply sets PKG_CONFIG environment variable pointing at a cross-pkg-config, $(GNU_TRIPPLET)-pkg-config but go tool doesn't support it. I've raised merge proposal to get that added https://codereview.appspot.com/104960043/</li>
<li>Next I just set GOPATH to where my packages are and CC as to which compiler to use</li>
<li>The last portion to the puzzle was to pash "-ldflags '-extld=$CC'" because the linker tool (5l) didn't use environmental variable CC and simply defaults to gcc. I'll raise a merge proposal for this.</li>
</ul>
<div>
Overall that's it. Given that all of above can be re-factored into standard variables (e.g. use $GNU_TRIPPLET prefix, and offer to override it), I see no reason why cross-compilation in go with cgo cannot eventually become a simple</div>
</div>
<blockquote class="tr_bq">
GOARCH=arm go build </blockquote>
<br /></div>
Dimitri John Ledkovhttp://www.blogger.com/profile/03781709847888802395noreply@blogger.com0