Emacsen's Bloghttps://blog.emacsen.net/2024-02-21T00:00:00-05:00Why I Tweak My Tools2024-02-21T00:00:00-05:002024-02-21T00:00:00-05:00Serge Wroclawskitag:blog.emacsen.net,2024-02-21:/blog/2024/02/21/why-i-tweak-my-tools/<p>I have a lot of productivity processes and tools. I have a tool for second brain, a tool for tracking my todo, a system for tracking files, a system for handling email, a tool for managing projects. More than that, I'm always tweaking the tools I have, looking to squeeze …</p><p>I have a lot of productivity processes and tools. I have a tool for second brain, a tool for tracking my todo, a system for tracking files, a system for handling email, a tool for managing projects. More than that, I'm always tweaking the tools I have, looking to squeeze a little improvement from them.</p>
<p>It's not out of self-indulgence that I do all this though, it's out of self-preservation. I have ADHD, and my ADHD holds me back. It's almost like a gravity, keeping me on the ground when I know I'm capable of flying. ADHD isn't the only neurological diagnosis I have; I'm also learning disabled, but ADHD effects me in my adult life more than anything else, and probably more than even I realize.</p>
<p>Because of that, I need things to help me, things like processes to help structure my activity, tools to aid me, medications when available, and people who can help me work through it.</p>
<p>One of the challenges of my ADHD is that things that may work very well one day may not work the next. That can be because my circumstances have changed, the need sof the situation have changed, or "just because". </p>
<p>Still, I'm always looking for way to improve, whether that means learning a new mental framework, trying a new tool, or learning a new process. I've been consciously working on my own productivity for twenty years, and I think I have things to share: tools that work for me, books that I've read which have helped, as well as general lessons on how I evaluate new tools and processes so I don't waste my time.</p>
<p>I'll be writing about my thoughts and insights into productivity tools. They'll generally be short posts focused on one thing. My goal isn't to get you do what I do, but to give some thoughts on finding or building the right system for you.</p>
<p>I hope you enjoy the posts and find them useful!</p>Reviewing the Framework Laptop2021-11-02T00:00:00-04:002021-11-02T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2021-11-02:/blog/2021/11/02/framework-review/<p>I've had laptops for about 25 years, from a variety of companies. My newest laptop is the one I'm using now, the <a href="https://frame.work/">Framework Laptop</a>. This laptop is one of two laptops I purchased recently that aim to be user serviceable, the other, the <a href="https://mntre.com/">MNT Reform</a> is a very different animal …</p><p>I've had laptops for about 25 years, from a variety of companies. My newest laptop is the one I'm using now, the <a href="https://frame.work/">Framework Laptop</a>. This laptop is one of two laptops I purchased recently that aim to be user serviceable, the other, the <a href="https://mntre.com/">MNT Reform</a> is a very different animal and not really comparable. This laptop is a much more traditional piece of equipment, very similar to my previous laptop, the XPS-13. I've owned several XPS13 laptops and have enjoyed them a lot, so the Framework is in for a challenge!</p>
<p>This review will be as unbiased as possible, but will emphasize my experience with the laptop, as well as an emphasis on my angle as a FLOSS enthusiast.</p>
<h2>Weight/Dimensions</h2>
<p>This is a small laptop, certainly in the same category as my old XPS13. It's a little larger but not substantially so, and weighs the same as the XPS.</p>
<table>
<thead>
<tr>
<th style="text-align: left;">Laptop</th>
<th style="text-align: left;">Width (inches)</th>
<th style="text-align: left;">Length</th>
<th style="text-align: left;">Height</th>
<th style="text-align: left;">Weight (lbs)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left;">XPS13</td>
<td style="text-align: left;">11.6</td>
<td style="text-align: left;">7.8</td>
<td style="text-align: left;">0.6</td>
<td style="text-align: left;">2.8</td>
</tr>
<tr>
<td style="text-align: left;">Framework</td>
<td style="text-align: left;">11.7</td>
<td style="text-align: left;">9</td>
<td style="text-align: left;">0.62</td>
<td style="text-align: left;">2.8</td>
</tr>
</tbody>
</table>
<p>The difference in the laptops' length is largely due to the difference in the screen, something I'll cover later in this review.</p>
<h2>Construction/Feel</h2>
<p>The construction of the Framework feels a bit plasticy, though not cheap. It has a bit of a hollowness to it that I feel is a bit unsatisfying, but this same feel can be found on other laptops, including my wife's Acer gaming laptop. It's also not present on Lenovo Thinkpads.</p>
<p>When holding the laptop, I can sometimes feel the plastic frame give a little under my fingers, and the machine sounds a little hollow.</p>
<p>I wonder how much of this feeling of plastic/hollowness is due to the nature of being repairable rather than other possible factors such as the type of material used.</p>
<p>This change is likely due to either different plastic or a lack of metal in some areas. It's a shame because it does make the laptop feel "cheaper".</p>
<h2>Keyboard/Touchpad</h2>
<p>The keyboard is decent, with good key press and some distance between the keys. The complaint I have is that as I press keys, there's a sense and a sound of hollowness from the device, which is unfortunate.</p>
<p>At the same time, the keyboard has decent key press, I notice a little more travel distance between this keyboard and my XPS-13. I feel a little more finger fatigue even while writing this, I think because of the feeling at the bottoming out of the keyboard.</p>
<p>The size of the keyboard is a little large, and this has been an issue for me since to get to certain keys I need to drag my palm, which leads to the touchpad getting pressed accidentally.</p>
<p>The touchpad is decent. It's quite large, and while that might be a benefit to some, for me the laptop's size means I can't type at certain angles because the laptop frame gets in the way. I also find myself running my palm into the touchpad sometimes. I hope this is just due to needing to adjust to a new layout, as it's happened more than once.</p>
<h2>Screen</h2>
<p>The screen is good, and it's clearly an area where Framework made some deliberate choices.</p>
<p>It's a 3:2 display with a resolution of 2256x1504. This display ratio sits in contrast to many laptops which are using a 16:9 ratio. Framework claims that this is better for coding and document preparation. After a couple of weeks, I think they're right that the increased vertical space has made coding and document preparation easier and that is a definite plus.</p>
<p>An unfortunate downside of the monitor size is that it's forced the bottom of the laptop to be larger, which is why I have the problem with typing at all angles.</p>
<p>It should also be mentioned that the Framework laptop has a glossy screen. While this isn't something that bothers me, I know that for some it's a dealbreaker. While one can purchase an after-market plastic screen protector for the Framework, the glossy screen may make it a non-starter.</p>
<h2>Speakers</h2>
<p>The speakers on this thing are not great. At lower volumes they sound tiny and muffled. While at higher volumes this is reduced, it's clear that speaker placement wasn't high on the priority list.</p>
<h2>Battery Life</h2>
<p>So far, the battery life on this laptop has seemed good. Six hours on a single charge is plenty good for me. I could probably optimize the laptop more if I were willing to to investigate some more power optimizations. This laptop is certain on par with the XPS13 and the System76 Lemur.</p>
<p>I do wish that it had an externally accessible battery pack, but that wouldn't fit with the slim design that this laptop offers.</p>
<h2>USB-C</h2>
<p>The Framework laptop, like the XPS13, uses USB-C ports that allow peripherals, but also allow the laptop to be charged from any port. This is an important feature to me and differentiates these laptops from other laptops such as the System76 Lemur laptop, which uses a round power connector and offers USB-C charging, but from only one port.</p>
<p>The USB-C feature also connects with the Framework "Expansion Cards", which I'll describe below.</p>
<h2>Expansion Cards</h2>
<p>The Framework laptop has several "expansion cards" which are swappable components that slide into housing in the laptop's side. They remind me of earlier laptop's PCMCIA cards, which could expand a laptop's functionality. In reality the Expansion Cards are nothing more than USB-C connected devices that slide into the laptop's housing, but by being so convenient, and USB being so fast, they serve the same purpose as PCMCIA, and are very convenient.</p>
<p>When I purchased my Framework, and at the time of this writing, the options for cards were somewhat limited, so I purchased three USB-C ports and one MicroSD expansion card. I am looking forward to having other cards in the future, namely a USB-A card and an HDMI card.</p>
<h2>Drivers/Linux/Free Software functionality</h2>
<p>The Framework laptop is clearly designed with Windows in mind, and when I ordered mine, I did not see a Linux or "No OS" version, though. I do not believe that it's possible to use a 100% Free/Open Source operating system without binary blob drivers on the Framework at this time.</p>
<h2>BIOS/UEFI</h2>
<p>Similarly, the Framework uses a proprietary UEFI and it does not support Coreboot/Libreboot.</p>
<h2>Repairability</h2>
<p>The most intriguing thing about this laptop is that the company behind it has stated that they stand firmly behind Right to Repair, a social movement that says we should be allowed to fix our own equipment. The laptop came with a screw driver and the website has a repair guide. More importantly, the company says it will create a marketplace where owners can purchase replacement parts such as keyboards, mainboards, and maybe most importantly, replacement batteries.</p>
<p>This is in stark contrast to Dell, which makes buying replacement OEM parts very difficult and discourages consumers from touching their computers, instead suggesting they buy new computers every few years just because a battery is starting to lose charge.</p>
<p>I stand behind the Right to Repair and feel it is complementary to the Free Software movement. While Framework hasn't been stellar about Free Software, I hope that this changes over time.</p>
<h2>Overall</h2>
<p>Despite some small issues with the layout and build quality, I like the Framework laptop. It was a good purchase and I hope to be using it for years to come!</p>
<p>I would like to see competitors to the Framework from more established folks in the Free/Open Source areas too. System76 is offering an open source desktop- they could surely offer a laptop that offers most or all of the same features, and also hasa greater emphasis on Linux/Free Software compatibility.</p>The Search for a FLOSS Mobile OS (Aug 2021)2021-08-23T00:00:00-04:002021-08-23T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2021-08-23:/blog/2021/08/23/floss-mobile-os-aug-2021/<p>For the last few weeks, I've been running <a href="https://calyxos.org/">CalyxOS</a>. It is the latest in Free/Open Source mobile phone operating systems that I've used. This post is a summary of my experience using FLOSS mobile OSes and what my experience can tell us not only about phones, but Free/Open …</p><p>For the last few weeks, I've been running <a href="https://calyxos.org/">CalyxOS</a>. It is the latest in Free/Open Source mobile phone operating systems that I've used. This post is a summary of my experience using FLOSS mobile OSes and what my experience can tell us not only about phones, but Free/Open Source OSes in general.</p>
<h2>The Search For a Mobile OS</h2>
<p>Calyx is not my first look for a FLOSS operating mobile OS, or even a handheld OS. I was a lucky owner of the Agenda VR3, a Linux-based PDA, as well as a iPAQ. I also owned a Neo FreeRunner, and while the device didn't work for me as a phone, I've always wanted a Free/Open Source phone.</p>
<p>In 2020 my hopes came true when multiple nearly-ready mobile OSes became either available or usable for regular use, and now I am the owner of a <a href="https://puri.sm/products/librem-5/">WLibrem 5</a> phone and <a href="https://www.pine64.org/pinephone/">PinePhone</a>, as well as having tried several Free Mobile OSes. To date I've tried <a href="https://ubports.com/">UBPorts</a>, <a href="https://postmarketos.org/">postmarketOS</a>, <a href="https://pureos.net/">PureOS</a>, <a href="https://mobian-project.org/">Mobian</a>, <a href="https://e.foundation/">/e/</a>, <a href="https://grapheneos.org/">GrapheneOS</a> and now <a href="https://calyxos.org/">CalyxOS</a>. UBPorts, postmarketOS, PureOS and Mobian are each different but we can roughly classify them as "Linux-based " and GrapheneOS, /e/ and CalyxOS are all Android-based. These are only rough classifications since the term "Linux-based" can mean anything that runs the Linux kernel and Android technically uses the Linux kernel. Nonetheless I'll be using this classification system for reasons that will become clearer in the next section.</p>
<h2>True Freedom?</h2>
<p>I know many people will have only a single criteria when evaluating these phones: Are they 100% Free as in Freedom, and the answer is that not all of them are, especially the Android-based OSes, which use non-Free binary blob drivers.</p>
<p>For some, this alone will be a showstopper, and I understand and appreciate that perspective, but I will review them anyway because for many people the issue of software freedom will take a back seat to other practical concerns, such as de-coupling their device from large companies such as Google or Apple. While I'd prefer an entirely Free OS, the immediacy of the threat to privacy means that I believe it is a justifiable position to place this practical need above the long term effort to have total control over our devices.</p>
<h2>To Android or Not?</h2>
<p>Calyx, /e/ and Graphene are all based on Android. They are the familiar Android operating system with software either modified or removed. This is in contrast to postmarketOS, Mobian, UBPorts and PureOS, all of which are not based on Android at all. Instead, these operating systems are more structurally similar to a "Linux-based" operating system that many geeks such as myself are familiar with.</p>
<p>The upside of this choice is that these OSes offer more familiar environments for Linux users and developers. They also are designed to run primarily on specialty hardware that is designed to make running and using them easy, sometimes without the need for any proprietary software, including proprietary drivers.</p>
<p>On the flip side, because there is no single unified development or GUI environment for these OSes, developers are forced to climb a steep hill of writing all new software for these phones. For example, it's only recently that "Visual Voicemail" has become available.</p>
<p>Worse still, programs that many rely on may not be available at all, even if they are Free.</p>
<p>While I want very much to be using one of these operating systems in the future, I could not recommend one of these devices for a "regular person" at this point, and for myself, they're not capable of serving as my daily everyday device yet.</p>
<p>I hope this changes, as I'd love to switch to a more traditional Linux-based operating system on my phone, but I can't quite make this leap just yet.</p>
<h2>Anbox</h2>
<p>Linux-using enthusiasts will no doubt be thinking "What about <a href="https://anbox.io/">Anbox</a>?"- the Android emulation layer that runs on top of Linux in a similar way that the popular program <a href="https://www.winehq.org/">Wine</a> allows Linux users to run Windows applications. To be sure, Anbox does exist, but I haven't had a chance to use it simply because the two phones I have that run Linux-based operating systems both suffer from problems. The PinePhone is a wonderful device, and at $150 is a steal, but it's fairly slow. The Librem phone cost over $700, but I find the hardware too clumsy and heavy to use as a phone.</p>
<p>Anbox would certainly be a benefit, as would a repository of mobile websites bundles as apps such as the Uber and Lyft websites. You can consider Anbox to be a complimentary technology to the discussion of the Android-based OSes.</p>
<h2>The Android Derivatives</h2>
<p>The first non-Google Android that many people used was CyanogenMod. You can read more about what happened to CyanogenMod <a href="https://www.androidauthority.com/cyanogenmod-lineageos-654810/">here</a>, but essentially this OS has been replaced with Lineage, which has in turn been forked off several times by other development teams, namely /e/, Graphene and Calyx.</p>
<p>In terms of usability, Calyx and /e/ are most similar, so I will talk briefly about GrapheneOS and what makes it different and unique. Graphene takes the position of being a "Security first" operating system. If there is an option to make something secure- ie through encryption or through application isolation, GrapheneOS appears to make the choice to support it. You can read about these features on <a href="https://grapheneos.org/features">their website</a>.</p>
<p>I ran GrapheneOS, and while I feel that I could likely make it work for me, it would have required quite a bit of tweaking. Graphene would probably be fine for a geek or developer, but it's not something I could recommend to a non-technical family member.</p>
<p>This leads to /e/ and Calyx. Before going any further, I want to say that I think both /e/ and Calyx are great operating systems and come very close to all of my goals.</p>
<h2>Google Services</h2>
<p>To understand both /e/ and Calyx, you need to understand a bit about how Android works and specifically how the "Open Source" Android operating system is a bit of a misnomer.</p>
<p>Android is indeed available under Free/Open Source licenses. This core operating system is called AOSP (Android Open Source Project), but most modern application rely on features that are not available through AOSP, including notifications, application debugging and other features and applications. These are called the Google Mobile Services, or <a href="https://en.wikipedia.org/wiki/Google_mobile_services">GMS</a>. Without Google Mobile Services, many Android applications simply won't work.</p>
<p>There are two general approaches to remedying this situation, and they're complimentary in nature. The first is to encourage developers to build software that does not rely on GMS, or replace GMS features with their FLOSS equivalents. This is the approach that the <a href="https://www.f-droid.org/">F-Droid</a> project takes. F-Droid is known to many Free Software enthusiasts as a software repository, but it also works with developers in building their software without reliance on Google through documentation and directly with developers on their forum.</p>
<p>The other approach is to allow unmodified software to run, but to replace the Google software stack with a FLOSS Software stack underneath. This means that even if a proprietary API call is made, it will simply trigger it's Free alternative instead. That's the approach that <a href="https://microg.org/">MicroG</a> takes.</p>
<p>Micro-G is not perfect however, as Google continually updates its APIs and does other checks as well, and the developers must try to catch up to a changing landscape. Nonetheless, by providing software to replace Google's API calls, it makes it far more possible to run unmodified Android Software.</p>
<p>Both /e/ and Calyx use Micro-G, and /e/ financially supports the project.</p>
<h2>Getting Apps</h2>
<p>While I would love to be a Free Software purist, we are often asked or even required to use proprietary software on our phones. This includes services which require apps, such as food or transportation, banking apps, or even government apps such as COVID tracking or passport apps.</p>
<p>Putting the issue of compatibility aside, it's often challenging to install the software itself. Android software is distributed in a format called APK, and it is possible to install APKs directly onto the phone, but it can be hard to find them since most websites simply point to the Google Play Store.</p>
<p>This is where the application <a href="https://auroraoss.com">Aurora Store</a> comes in. Aurora is a Google Play Store client. It essentially pretends to be the Google Play application, but is built with privacy in mind.</p>
<p>You could use it to download your paid applications, or you can use it in anonymous mode to find/download/update your existing programs.</p>
<p>It does come with a catch... Using Aurora Store could be considered a violation of the Google Terms of Service and Google could terminate your account for it.</p>
<p>While for some people this might not sound like a big deal, it could be very serious for others. Having your account terminated would mean that Google would lock you out of your email, your calendar, your Youtube, Google Payments, or any other Google services. If you use your Google account in an official capacity, such as for government forms, it could potentially make your life very difficult.</p>
<p>It's because of this risk that while I think Aurora is neat, I hesitate to recommend it to non-technical friends- especially if they would log in with it.</p>
<p>Calyx comes with Aurora Store, while /e/ does not.</p>
<h2>Contacts, Calendar, Settings and Backup</h2>
<p>When you install a Google or Apple phone, it requires you to log in. Apparently the same is now true for a Windows 11 computer- you must either create or log into an existing account in order to use it.</p>
<p>For FLOSS fans, or anyone who cares about our privacy, this is an anathema, but there is something to be said for the ability to start a new phone and have all your contacts, your calendar and your settings all "just work" out of the box. While I want to say that this isn't a big deal, the fact is that not having these features seems like a step backwards.</p>
<p>This is where Calyx and /e/ differ. The /e/ Foundation has created a server that users can create an account on during installation and lets you sync your contacts, and possibly do phone backups as well. Calyx does not do that out of the box, though it is possible to achieve with some software.</p>
<p>This is a big plus for /e/. For users who don't entirely trust the /e/ Foundation, the software they use is essentially just Nextcloud, so it would be possible to self-host as well.</p>
<p>Syncing up contacts and calendar on Calx involved installing the Nextcloud client, configuring it for my Nextcloud instance, then installing <a href="https://www.davx5.com/">DAVx5</a> and then configuring it to use the Nextcloud client. This wasn't difficult for me, but it's enough of a pain that I think a non-technical person might find it challenging.</p>
<p>Moreover, while I run a Nextcloud server, I certainly don't think that most people have their own Nextcloud instance or one that they can just use, so kudos to /e/ for making this process as easy as it could be.</p>
<h2>Calyx's Nifty Privacy Features</h2>
<p>Where /e/ shines in terms of easy of use, and Graphene shines in terms of security, Calyx has made wonderful choices in terms of privacy. As a small example, when you make a normal (non-encrypted) phone call, it reminds you that the conversation and your location are not protected. These small touches make the OS feel both secure and also "fun". It came pre-installed with F-Droid, Signal, Tor, Briar, as well as other privacy apps. Even the default wallpaper, proudly declaring "I never Metadata I didn't like." made me feel like I was back in the late 90s when running Linux was in itself a sort of act of rebellion.</p>
<p>Calyx also comes bundled with its own VPN. I'm not sure how it is that Calyx can install a VPN but not have the resources or interest of running its own Nextcloud ala /e/.</p>
<h2>Maps, Location and Geolocation Services</h2>
<p>Maps and Geolocation services are probably the biggest show-stopper in terms of ease of use of these phones for most people. This is a topic I know about in some detail, having been involved in the OpenStreetMap project. I won't rehash the issues in detail, but I think there are three large impediments for most people.</p>
<p>The first impediment is that Google and Apple both provide a geolocation service as part of their phone operating system. Another way to say this is that Google and Apple can figure out exactly where you are. To do this, they use a combination of several pieces of information- everything from GPS signals, cell tower triangulation, WiFi hotspots and more. Using this combination of data, they are able to accurately pinpoint your phone's location quickly and easily.</p>
<p>Without a Geolocation service, the phone must try to figure out its location on its own, often using GPS. Unfortunately GPS is slow, somewhat inaccurate and drains battery quickly. There are attempts at creating a Free geolocation service, such as the <a href="https://location.services.mozilla.com/">Mozilla Location Service</a>, but as far as I know, none of these services are ready for prime-time use.</p>
<p>Both Google and Apple have a map platform. In the FLOSS world, we have OpenStreetMap. Google/Apple maps provides not only maps, but also a robust and complete Point of Interest (POI) map, detailing store information, etc. OpenStreetMap's street maps are often quite good, but POI data often falls by the wayside. This means that looking up the nearest gas station or restaurant can be a hit-or-miss experience. Getting more contributors to OSM would solve this, as might some kind of web scraper that could contribute to a POI database, but in the meantime this could make OSM based maps a non-option.</p>
<p>Lastly, OSM maps do not contain real time traffic data and so routing may be significantly worse than it would be for Google Maps.</p>
<p>If OSM maps are bad, why not switch to Google Maps for short periods? This is certainly possible using the website. As for the Google Maps app, in my experience the app either did not run, or it ran and complained about Google services not being available, while still working. While I found this acceptable for me, it might be too big a leap for users.</p>
<p>This isn't a phone issue as much as it is an issue of geolocation and geographic database services, but because maps are such a large part of the phone experience, it's worth mentioning.</p>
<h2>Ease of Install</h2>
<p>One of the biggest challenges with getting non-geeks to use Free technology is the install process. Back in the 90s, avid Linux users such as myself ran Installfests, gatherings where people would bring their computer and get help installing Linux on it. Those days are over, but I can still imagine someone finding the install process for these OSes to be intimidating.</p>
<p>/e/ takes care of that factor by offering its OS pre-installed on certain devices. Unfortunately for me, they currently only sell their phones in Europe.</p>
<p>Luckily for us, many Free OSes are taking a cue from Google and offering a web based installer. GrapheneOS currently offers a web based installer, /e/ is slated to soon, and while Calyx doesn't, their installer was also fairly straightforward. Nonetheless, the easier the install process could be, the better.</p>
<p>Calyx and Graphene also only support a limited number of phones, which makes it easier to support a better install experience.</p>
<p>One downside of all the Free Android OSes I've tried is that the phones complain about a non-Google OS on boot in a somewhat intimidating message.</p>
<p>Installing the Linux-based OSes is much easier, but is still a multi-step approach. While I love being able to install an OS by inserting an SD card, eliminating the multi-stage install process would be even better.</p>
<h2>What OS would I choose, and thoughts on the desktop</h2>
<p>As of the time of writing (August 2021), I would choose an out-of-the-box supported /e/ phone first, but if that wasn't available, I would install Calyx without hesitation</p>
<p>That said, I think that the extra steps in Calyx to sync contacts, calendar and settings make it a non-starter for non-geeks. Users today expect that applications work without additional configuration steps, and it's hard to blame them. When I first started using Linux in 1997, I was amazed at how much software for computer came bundled on CDROM. In 1999, I switched to Debian and it felt like I had the entire world at my fingertips compared to Windows users who would need to either purchase software, or at least go through the manual installation process. I could reinstall and reconfigure a desktop in just a few hours.</p>
<p>Now these kinds of installation experiences are the norm, but we're still expecting Desktop and mobile users to spend hours configuring applications, and that's assuming they have a resource like Nextcloud available to them- most don't.</p>
<p>That means one of two things need to happen, and likely both. The E Foundation has taken a page out of Google and Apple's playbook by offering synchronization out of the box. It might be that large FLOSS providers begin offering services alongside their software. This would not be unprecedented, as Mozilla offers several services, and everyone relies on Wikipedia and even OpenStreetMap services. It may be that this becomes just another cost of being a FLOSS project.</p>
<p>Another possibility is the creation of tools designed around cooperative storage. That's the ideas of the Datashards project, though we haven't made any announcements lately. I want to see storage systems like Nextcloud replaced by cooperative storage systems that handle synchronization and backup transparently and allow your devices to work together without the need for additional software.</p>
<h2>Final Thoughts</h2>
<p>My experience shows we're very close to having a Free/Open Source phone, but we need to make a few sacrifices along the way. We as geeks could certainly use such a system, especially if we care about privacy, but in its current state, I'm not sure I can recommend the current generation of mobile operating systems for non-geeks, even if they are looking for a de-Googled experience.</p>
<p>I think we can also look at the issues of backup and synchronization to improve our desktop OS experiences as well. I believe that when FLOSS Software achieves 80% of what its proprietary competitor can do, it becomes a tipping point. I think we're at 70% right now. The other 10% is very achievable, but it will be hard work.</p>Stern Fan In Recovery2021-07-03T00:00:00-04:002021-07-03T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2021-07-03:/blog/2021/07/03/stern-fan-in-recovery/<p>Hello everyone. My name is Serge, and I am a recovering Howard Stern fan.</p>
<p>I feel like I'm at an AA meeting when I say it, but it feels oddly good to get it off my chest. Howard Stern has been a significant figure for roughly half of my life …</p><p>Hello everyone. My name is Serge, and I am a recovering Howard Stern fan.</p>
<p>I feel like I'm at an AA meeting when I say it, but it feels oddly good to get it off my chest. Howard Stern has been a significant figure for roughly half of my life, and only in the last year or so have I really begun to understand the influence of the show on me, the kind of person Howard Stern really is, and how that plays into my own past trauma and pathology. This has allowed me to understand myself better and heal in new and significant ways. But I'm getting ahead of myself- let me start at the beginning.</p>
<p>I had a pretty lonely childhood. I didn't have siblings. I went to "special school" (ie "Special Ed") and my parents did a number on me emotionally. Both my parents learned early on that they could use fear of abandonment as a tool to control me, so they both cultivated it. At age eight, my father threatened me that he would quote "Come to my school and tell the other students all of my secrets". As an adult now, I don't know what kind of shameful secrets my father would have told my third grade classmates, but at the time the idea was scary and filled me with shame.</p>
<p>This type of threat was commonplace in my home. Mirroring it, when I was seventeen, my mother said that she hoped my girlfriend would quote "Find out what a terrible person I really was" and leave me.</p>
<p>I grew up being told that no one would want to be my friend and that friends I did make would "learn who I was" and wouldn't want to be around me anymore.</p>
<p>At around age 16, I first heard Howard Stern on the radio. I now know it was during "The News", a segment on the show when Howard's assistant Robin would read out the day's news and Howard would comment.</p>
<p>"Adopted kids will kill their parents.", I heard the deep, manly voice on the radio exclaim.</p>
<p>"Not all adopted children, Howard.", I heard the woman on the radio respond, chuckling.</p>
<p>"Yes Robin, all of them. All adopted children will kill their parents."</p>
<p>It was so extreme as to be absurd. It was trolling, years before I would know the word.</p>
<p>On an interview I saw on the news, Howard made a similarly absurd statement about cybersex on Prodigy chat rooms, explaining to the interviewer that in his new book, he had transcribed his experiences with online sex chatrooms, but that if the interviewer asked his wife, Howard would tell her that this was all fiction, quote "a bit for the radio"- but that we, the audience, knew the real truth.</p>
<p>As an Andy Kaufman fan, I saw it as an extension of that style of comedy- of breaking the fourth wall and provoking a response.</p>
<p>I was in love.</p>
<p>I stayed in love as I continued to listen and heard radio bits such as when Howard punished received low ratings in Dallas and in retaliation, decided to "punish his audience" with bad music, or when he went on the air after supposedly "saving a suicidal man", Howard planned a press conference on the air, planning out each word and even the responses of others around him. We, the audience, were in on every moment, and so when Howard then held his press conference live on the air and we heard the press conference go off exactly as planned, including a seemingly impromptu conversation between Howard and Robin about whether or not he should be called a hero. I was infatuated.</p>
<p>Howard played a huge role in my life from then on. I would wake up at 5:45 in order not to miss the show. In college I programmed my computer to tune the radio to the local Stern station, record the show onto my hard drive and then encode it into an mp3 that I would sync to my portable mp3 player. This was in 1998, three years before the launch of the iPod and six years before the term "podcasting".</p>
<p>I kept this setup throughout college and beyond. I listened to the Howard Stern show almost every day. On the day he left terrestrial radio for satellite, I listened live at work and had to excuse myself to the bathroom not to let my coworkers see me cry.</p>
<p>I moved with him to Sirius radio and just as Howard had promised, the show was better than ever. Not only did Howard seem invigorated, but the program director, Tim Sabean, had transformed the Howard Stern show from a single show to an entire galaxy of programming, including The Wrap Up Show- a show by Stern show staffers about the day's show, the Intern Show, a show about the Stern show from the perspective of show interns, to wackier shows like the Riley Martin Show, a radio show about space aliens hosted by an occasional guest who claimed to have been abducted by aliens and brought back to earth to share a message of both peace and warnings, and to sell crudely drawn pictures that would be used as tickets to get the owner on an intergalactic space ship.</p>
<p>Howard had created a larger version of what his show had been to me- a way of feeling like I was part of a group- of being on the inside, and now there was more content than even I could consume.</p>
<p>For a lonely kid who didn't feel safe at home or have many friends, the Stern Show felt like more than home, it felt like family. And that's no wonder, as I was listening to the show nearly every day, absorbing each morning discussion of Stern and other staffer's personal life, show bits, celebrity interviews, wack pack segments, and the news. I listened to the show far more than I spoke with my actual family, and in turn I felt like I'd learned life lessons from the show, especially as they related to early Stern bits such as remaining faithful to one's spouse, virtues around not being hypocritical, and speaking truth to power, even if it made you less popular.</p>
<p>I didn't always agree with everything the show did. I didn't care for the strippers and I didn't like that Howard made fun of people who were different or disadvantaged such as black people, homosexuals or transgender people, but I explained away those problematic bits as being either about the times the show aired, satire, or an artifact of Howard's age and generation. Like a favorite uncle, I didn't have to agree with everything he said to love him.</p>
<p>One day my boss asked me how many hours a week I listened to Howard Stern. I did the math and calculated that I listened around twenty or twenty-five hours a week. He said "...That's got to affect your mind."</p>
<p>He was right. It did.</p>
<p>The show felt <em>good</em>. It felt fun and even loving in a way.</p>
<p>I saw myself in Howard- a geeky Jew who was misunderstood and rebelled against the system. I didn't think Howard would necessarily like me if he met me but I could at least have someone to look up to.</p>
<p>When long time show writer Jackie Martling left and Artie Lange came in, it felt strange in a way that I imagine it feels when a mother brings home a new boyfriend. But in time I came to accept him into the family as well, a lovable lug from New Jersey, the same state as I was from.</p>
<p>Similarly, I felt odd when Star Trek's George Takei became the show's guest announcer and sit-in guest, talking in candid detail about his past and present sexual activity, his sexual schedule (Sunday is sex day) and where and how he preferred to masturbate. It was odd, but I knew more about this actor's sexual life than I did about my neighbors or even many of my friends.</p>
<p>Once Artie left the show in 2009, the tenor of the show changed. It felt emptier and slower. I thought that just as the show had gone through a lull after Jackie left, that the show would bounce back once again once it found its rhythm and pacing, or possibly a new cast member to fill the role.</p>
<p>But that didn't happen. Instead, the show just became slower and more repetitive, more structured and less spontaneous. Staff drama had always been a staple of the show, but now it felt more over-hyped and manufactured.</p>
<p>Some time around 2015 or 2016, I realized I didn't care any more and I stopped listening to the show. It wasn't a conscious decision at first. I would go days without listening and then listen to a show. At some point I just stopped listening and didn't really start up again.</p>
<p>I still listened to old shows. I enjoyed "Classic Stern". I found old shows online and replayed them.</p>
<p>Every once in a while I would find a copy of a new show someone had posted online and give it a listen. Sadly the luster on the show had worn off and I retreated back to old clips from classic moments on the show.</p>
<p>But something changed in the last year.</p>
<p>After the pandemic hit, I was alone. I found myself isolated from my fiancee and my pets by a national border. I was alone in my apartment for about fifteen months. During this time, I also had a very bad ending to something that spanned friendship, collaborator, and planned business partner. The word relationship is often used to mean a romantic relationship, but this relationship felt as close as it could be without any romantic or sexual feelings.</p>
<p>The relationship ended because I became that the person I was collaborating with was a narcissist. As I began to ask for solid commitments in exchange for my free work and began to assert boundaries, the interactions became more toxic until I had to end the relationship altogether.</p>
<p>This was sadly not my first time at the rodeo. My childhood has left me with a lot of scars and codependency, or what psychologist Ross Rosenberg calls "self-love deficit". Simply put, I am always afraid of being rejected, and especially susceptible to people who are affectionate. This makes me especially vulnerable to people with Cluster-B personality types, including Borderline and Narcissistic Personality Disorder.</p>
<p>But back to Stern...</p>
<p>I'd been interested in the experiences of other listeners and also looking for updates about Artie Lange, who was no longer part of the show. Through that journey, I began to hear about horror stories such as when long time Stern show engineer Scott Salem asked Howard if he could raise money to try to cure his wife's cancer and in response, Stern had him first demoted, then fired.</p>
<p>More than that, other former staffers began coming out to talk about their experiences working on the show including draconian rules such as not being allowed to greet Howard in the hall, no communication with, or even about ex-staffers, having their jobs kept in limbo for extended periods, as well as a pattern of severe underpayment accompanied by insults or humiliation about the notion of leaving. While on-the-air humiliation was a staple of the show, I'd always assumed it was countered with a positive working environment off the air, but the pattern of mistreatment of staffers was consistent and complete- Howard mistreated his staff, and always had since the early days.</p>
<p>As I learned more about the show's inner workings, my feelings turned from disappointment to anger. How could Howard treat his staff so badly? How could he humiliate them both on and off the air? And how could I have been either blind to it or make excuses for it all of these years?</p>
<p>All of this has lead me to "Quite Frankly: A Howard Stern podcast", hosted by two former Stern fans, Jim and Samantha. These two hosts, sometimes joined by guests, take apart old Stern shows and analyze them critically. They deconstruct the show's contents and show Howard's truth distortions, lies and manipulation. They also bring on expert guests to analyze the show through lenses such as psychology, and Narcissistic Personality Disorder and how we as an audience can identify these traits.</p>
<p>I've listened go nearly a dozen of their shows, which now number nearly a hundred, and while I don't always agree with the hosts or their presentation, I understand it. Both Jim and Samantha seem angry, genuinely angry at Howard and at the show, which is understandable...</p>
<p>Howard garnished not just listeners, but superfans, people like myself who would listen to the entire show, start to finish, buy not only his books, but those of his staffers as well, and continue to do so for years, even decades. It makes sense that when faced with such fanaticism, those same people don't turn the show off, when they realized they've been suckered, they feel angry and betrayed, and that is exactly how it is with the two hosts. The two hosts clearly know their Stern history as only superfans do and yet there's dismissiveness and derision in their voice when talking about the man himself.</p>
<p>Listening to this pocdcast feels like therapy, or at least catharsis, not only for the love I had for Howard Stern and his show, but also for the ways that Howard used his show and his image to pull the wool over our eyes.</p>
<p>Through it, I feel myself healing from the experience of a twenty year relationship that came at a vital time in my life and shaped who I am and how I saw the world. I see not only the patterns in Howard, but also the patterns in past relationships, of a former colleague who still uses their fame and persona to garner a fan base, and even sometimes in ways that I see myself reflected in Howard's behavior.</p>
<p>I asked my therapist once why we talk about the past in therapy. He said that while we can't actually go back in time that we can use therapy to find a new way to relate to past events and bring an awareness and understanding that we didn't have the first time. Listening to Quite Frankly feels like therapy, and I feel like I'm gaining a deeper awareness of myself than I ever have. The old Stern show bits still have a place in my past and my heart. Like my dysfunctional childhood, I'll always love them, but now I cringe at the way that Howard bullies and humiliates his staff, manipulates the callers and railroads his guests. The show makes me cringe. This mix of affection and cringe is good. It's healthy. It lets me know that I'm healing and learning. It's all anyone can do.</p>
<p>If you're an ex-Stern fan, I hope you are joining me on this journey of understanding. If you still love Stern, then the words I've said probably have no impact. And if you're someone who never liked Stern, I hope that this has given you insight onto why I did and the way that he and his show gave me a sense of belonging when I needed it most.</p>
<p>Now, at 42, I'm moving on and making my own family and community.</p>
<p>Here are some links that have opened my eyes:</p>
<p><a href="https://www.reddit.com/r/howardstern/comments/m6plcq/timeline_of_the_stern_show_decline/">Timeline of the Stern Show Decline</a></p>
<p><a href="https://nypost.com/2021/04/27/howard-stern-has-lost-his-sting-and-his-mojo/">An article in the New York Post about the quality of the Howard Stern Show</a></p>
<p><a href="https://www.city-journal.org/howard-stern">What Happened to Howard Stern?</a></p>
<p><a href="https://www.youtube.com/watch?v=iguOZUlSjCE">Artie Lang goes on Opie and Anthony and talks about what happened between him and Howard</a></p>
<p><a href="https://www.youtube.com/watch?v=aLArbG5beFU">Jackie, Billy West and Stuttering John talk about their experiences on the Stern Show</a></p>
<p><a href="https://fillmorefingers.podbean.com/">Quite Frankly: A Howard Stern Podcast</a></p>The Value of Outstretched Arms in Free Software2020-04-12T00:00:00-04:002020-04-12T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2020-04-12:/blog/2020/04/12/value-of-outstretched-arms-in-free-software/<p>In 2004, I was sitting in my living room watching one of my favorite programs- <a href="https://en.wikipedia.org/wiki/Frontline_(American_TV_program)">Frontline</a>- a PBS show doing investigative journalism on a variety of topics.</p>
<p>This particular episode, <a href="https://www.pbs.org/wgbh/pages/frontline/shows/persuaders/etc/script.html">"The Persuaders"</a> was about advertising was not only changing the way we buy but the way we think. As part …</p><p>In 2004, I was sitting in my living room watching one of my favorite programs- <a href="https://en.wikipedia.org/wiki/Frontline_(American_TV_program)">Frontline</a>- a PBS show doing investigative journalism on a variety of topics.</p>
<p>This particular episode, <a href="https://www.pbs.org/wgbh/pages/frontline/shows/persuaders/etc/script.html">"The Persuaders"</a> was about advertising was not only changing the way we buy but the way we think. As part of this discussion, they examined cults- including the Hare Krishnas to "cult brands" like Apple.</p>
<p>As an example, they showed a person talking about Linux and how some people were "part of the tribe". While they never showed the person's name, I recognized them and that moment struck me. It was like I was seeing myself in the mirror- I had spoken that way in the past, but seeing it in front of me, I realized I never wanted to look or think like that in the future.</p>
<p>Being part of a community is important. I've spoken in the past about how being part of the Free Software movement literally saved my life. Being part of the this movement also included breaking free of terms like <a href="https://www.gnu.org/philosophy/not-ipr.en.html">"Intellectual Property"</a>, which has an impact on the way we think about these topics. It's also meant that I've discouraged the use of proprietary formats such as Microsoft Office, when the <a href="http://www.opendocumentformat.org/aboutODF/">OpenDocument Format</a> is both available and standardizes.</p>
<p>At the same time, I've also seen people inadvertently use Free Software to divide or shame people. If someone uses the "wrong word" (for example Open Source instead of Free Software or Linux instead of GNU/Linux), or admitting to use proprietary software, they may get an earful from someone in the Free Software community.</p>
<p>I've thought a lot about why this is and my conclusion is there are three reasons why Free Software advocates become this intense about terms and phrasing. The first is that as we've learned about the ways that our society has indoctrinated us into thinking about these topics (including the idea that copyright is paramount to property), that we're motivated in the same way to help others break free of the mind-control that we were under. We want to liberate them the same way we ourselves were liberated.</p>
<p>The second reason is less altruistic but I think sadly just as true, which is that these verbal signals are part of, as the "Linux user" in Frontline said. When I got my bachelors in Psychology, I learned about the idea of cognitive dissonance and <a href="https://sites.psu.edu/alyssaboobpassionblog126/2018/09/27/hazing-and-its-unhealthy-relationship-with-cognitive-dissonance/">how it makes us love things we sufferfor</a>. I believe that some of our strong reactions are part of this unconscious desire to "bring people into the fold", doing the same kind of thing that was done to us.</p>
<p>Lastly and possibly most importantly, while such people are highly disruptive and hurtful, in reality, they represent a small minority of the community.</p>
<p>When I started the <a href="http://librelounge.org">Libre Lounge</a> podcast with my friend <a href="https://dustycloud.org">Chris Webber</a>, one of my goals was to widen the umbrella and embrace more people into Free Software with open arms. We want to bring new people to Free Software and help them see that we are a warm and caring community.</p>
<p>In doing that, we've talked about a variety of topics, worked hard to bring on guests with varying backgrounds, connected larger cultural movements to our own, and generally tried to retain the sense of fun and playfulness that we think is so important in maintaining a healthy community.</p>
<p>Occasionally disputes arise around terminology and in those moments, I'm reminded of the <a href="https://www.theguardian.com/stage/2005/sep/29/comedy.religion">old joke</a> by Emo Philips:</p>
<blockquote>
<p>Once I saw this guy on a bridge about to jump. I said, "Don't do it!" He said, "Nobody loves me." I said, "God loves you. Do you believe in God?"</p>
<p>He said, "Yes." I said, "Are you a Christian or a Jew?" He said, "A Christian." I said, "Me, too! Protestant or Catholic?" He said, "Protestant." I said, "Me, too! What franchise?" He said, "Baptist." I said, "Me, too! Northern Baptist or Southern Baptist?" He said, "Northern Baptist." I said, "Me, too! Northern Conservative Baptist or Northern Liberal Baptist?"</p>
<p>He said, "Northern Conservative Baptist." I said, "Me, too! Northern Conservative Baptist Great Lakes Region, or Northern Conservative Baptist Eastern Region?" He said, "Northern Conservative Baptist Great Lakes Region." I said, "Me, too!"</p>
<p>Northern Conservative†Baptist Great Lakes Region Council of 1879, or Northern Conservative Baptist Great Lakes Region Council of 1912?" He said, "Northern Conservative Baptist Great Lakes Region Council of 1912." I said, "Die, heretic!" And I pushed him over.</p>
</blockquote>
<p>When we argue about who is "more pure" or when we tell people that they're bad or evil because they don't use either the same software stack we do, or use the same terminology we do- then we've lost the point of Free Software, which is to spread Freedom.</p>
<p>Be the person who welcomes, not the one who shuns.</p>The DNC is Really Talking To Itself2020-03-01T00:00:00-05:002020-03-01T00:00:00-05:00Serge Wroclawskitag:blog.emacsen.net,2020-03-01:/blog/2020/03/01/the-dnc-is-really-talking-to-itself/<p>Listening to the news about the Democratic party can be disheartening at best. This week a story in the New York Times came out discussing how <a href="https://www.nytimes.com/2020/02/27/us/politics/democratic-superdelegates.html">DNC leadership is willing to disenfranchise up to half the party</a> in order to prevent Bernie Sanders from getting the nomination.</p>
<p>They claim that …</p><p>Listening to the news about the Democratic party can be disheartening at best. This week a story in the New York Times came out discussing how <a href="https://www.nytimes.com/2020/02/27/us/politics/democratic-superdelegates.html">DNC leadership is willing to disenfranchise up to half the party</a> in order to prevent Bernie Sanders from getting the nomination.</p>
<p>They claim that this is in order to solidify a win. They claim that it's the <a href="https://fivethirtyeight.com/features/voters-wholl-support-biden-but-not-sanders-probably-really-do-mean-it/">swing voters that they're courting</a> and that those voters would never vote for Sanders. It's policies, they claim, or occasionally they'll claim it's those mid-westerners and their anti-Semitism, usually while engaging in <a href="https://fair.org/home/its-media-not-bernie-sanders-that-have-an-antisemitism-problem/">anti-Semetic tropes</a>.</p>
<p>Meanwhile, On the Media put out a story this weekend about the <a href="https://www.wnycstudios.org/podcasts/otm/segments/new-theory-swing-voters">disenfranchized progressive voter</a>, just how many progressives are turned away from voting, or vote for a third party rather than vote for a moderate.</p>
<p>On its face, these two situations don't reconcile. The Democratic Party must want to in, mustn't it? Instead of courting Republicans who might somehow be persuaded to vote for a Democrat (despite Trump's 80% approval rating amongst Republicans) why wouldn't they work to energize the voter base- to register more underprivileged, undercounted, underrepresented people and energize the youth?</p>
<p>Why wouldn't the DNC want to show the country that Trump is wrong in his "Do nothing Democrats" taunt, that the Democratic Party does have a grand vision as a counter to the grand vision of Republicans?</p>
<p>The answer is simpler than it seems... The DNC's fear-mongering about Sanders not being a viable candidate is not for Republicans or the moderates amongst its ranks, but rather they themselves.</p>
<p>We see this reflected not just in political circles, but the corporate "liberal media" where Sanders is <a href="https://www.jacobinmag.com/2019/11/corporate-media-bernie-sanders-bias-msnbc-warren-biden">consistently painted in a negative light</a> even on self-described <a href="https://www.wnycstudios.org/podcasts/otm/episodes/msnbc-being-very-very-calm-about-bernie-sanders">liberal news outlets</a>.</p>
<p>The fact is that the critique that many Republicans have had over the hypocrisy of the Democratic party is real. There is a "Limousine Liberal" with a vested interest in the status quo, who decries Trump's "Make America Great Again" slongan, but who pines for the days of the Clinton era, where public programs were cut, but since corporate growth was high, only poor/brown people noticed.</p>
<p>Sanders makes the DNC uncomfortable because he forces the Democratic party to come face to face with the reality that it isn't for poor people, brown people or the youth, but rather to keep things simmering just enough below the surface to keep the lid from popping off.</p>
<p>With Trump in office, the lid has popped off and now the DNC leadership is scrambling to figure out how to keep control of the narrative. They've invented a make-believe voter, a Joe or Jane Republican who watches Fox News but will be persuaded to vote for a "moderate" Democrat.</p>
<p>It's time for the DNC leadership to get honest with itself and the American people. The Democratic coalition is breaking apart at the seams. The party is split between two very different ideas, one where we <a href="https://www.youtube.com/watch?v=U4hShMEk1Ew">dream of the 90s</a> and the other where we live in the present and present the people with a comprehensive plan to enact sweeping changes that will save our children, help heal our environment and repair our decaying infrastructure.</p>
<p>I've lived through the 90s and I don't want another Bill Clinton. I want another Franklin Rosevelt.</p>Thoughts on Canonical S-Expressions2019-10-02T00:00:00-04:002019-10-02T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2019-10-02:/blog/2019/10/02/thoughts-on-cannonical-s-expressions/<p>Datashards currently uses Canonical S-Expressions as a data format and after using it for a few months, I have some thoughts.</p>
<p>First things first: If you aren't familiar with the format, let me give you a quick rundown. Canonical S-Expressions are a bit like regular S-Expressions, with a twist. If …</p><p>Datashards currently uses Canonical S-Expressions as a data format and after using it for a few months, I have some thoughts.</p>
<p>First things first: If you aren't familiar with the format, let me give you a quick rundown. Canonical S-Expressions are a bit like regular S-Expressions, with a twist. If you already know Lisp, none of this will be new, but for the rest of you, there are two items in an S-Expression- a list and an atom. A list is what it sounds like- a sequence of things. And an atom is a thing. An S-Expressions looks like:</p>
<div class="highlight"><pre><span></span><code>(item1 item2 item3 item4)
</code></pre></div>
<p>If you're familiar with Python or Javascript, you can think of that as the same as:</p>
<div class="highlight"><pre><span></span><code>[item1, item2, item3, item4]
</code></pre></div>
<p>In Canonical S-Expressions (csexp), every atom is actually a byte object, and we say the size of the byte object by prepending it with the number of bytes, followed by a colon:</p>
<p>(5:hello5:world)</p>
<p>That's a list of two items, 'hello' and 'world'. I'm putting these in quotes but the values aren't strings, they're bytes. That means it's very efficient to put raw binary data in a csexp. If you put binary data in JSON, you'd have to do something like base64 encode it. No need in csexp!</p>
<p>You can also give a "type hint" in csexp, so if you have a binary object that represents an image, you can stick the mimetype in the csexp, such as:</p>
<div class="highlight"><pre><span></span><code>([image/jpeg]1024:<bytes>)
</code></pre></div>
<p>You can also store other lists inside of a csexp, such as:</p>
<div class="highlight"><pre><span></span><code>(9:groceries(4:milk5:bread))
</code></pre></div>
<h3>What I Like</h3>
<p>There's a lot to like about Canonical S-Expressions. They're extremely space efficient, very flexible and super easy to parse. Writing a reader for a csexp is fairly trivial. And even if your language doesn't already have a csexp library, you can easily write one in a day, if not an afternoon.</p>
<p>The other thing I like about Canonical S-Expressions is that they do what they claim to do and nothing else. They're a binary format that only does byte strings and lists.</p>
<h3>What's Not to Like About Canonical S-Expressions</h3>
<p>Working with CSEXP data can be a pain. You're always stuck writing a reader for your data. Your reader will take the resulting abstract parsed data and convert it into something your application will actually consume. In some cases this conversion is easy, 3:100 becomes the integer 100. If you want to store more complex data structures, such as associative arrays, however, then you'll need to think about it.</p>
<p>Since CSEXP doesn't have associative arrays, only lists, you'll end up writing the serialization/deserialization format on your own. You could store them as lists of lists, ((key val) (key val)) or the more compact form of (key val key val) or you could (ab)use the hint system, such as ([key]value [key]value). Whatever choice you make, it will be specific to your application and someone who reads the document will need to think about the choices you made beforehand. Or if you're inheriting data in this format, you may end up having to guess at the meaning of the data structure.</p>
<p>This type of step is necessary for many serialization formats. In some, like Protobufs, it's a requirement. In XML, it was not strictly necessary but almost always done, and in some applications using JSON, it may not be necessary at all.</p>
<p>Canonical S-Expressions occupy a strange middle ground where having a formal schema is not strictly necessary, as it's schema-less, but it's also challenging to work without one.</p>
<h3>Flexible (Schema-less) Data Serialization Formats</h3>
<p>Flexible data formats are a topic of deep discussion and debate. In the 90s, it seemed that the world had converged on XML as the One Format to Rule Them All. The problem with XML is that even though the format is self-documenting in some ways, ie <tag></tag>, the value inside tags needed to be converted during a secondary reader, separate from the parser.</p>
<p>Since this distinction isn't always clear, the parser parses the raw data into a machine readable data structure, while a reader parses the data (usually post-parsed) into application specific data structures.</p>
<p>Canonical S-Expressions have the same problem in regards to needing a reader that XML does, but unlike XML, you don't have the storage or bandwidth issues of the tags.</p>
<p>JSON seems to have won out the generic data format wars by offering some types, making writing a reader trivial (or in some cases, unnecessary) but anyone who has ever had to work with JSON knows that its thin layer of types is misleading. As an example, "How do you store a date in Javascript?"</p>
<p>You could store it as Unix time, seconds after the epoch, or you could store it in an ISO 8601 formatted string, ie "2008-09-15T15:53:00+05:00" or an RFC 822 date format, or something else entirely. Your parser will happily give you a string, but you're stuck needing a reader to do that final conversion, just like you did with XML.</p>
<p>JSON-LD solves some of this by giving your values semantic meaning, but it makes the parser more complex.</p>
<p>And neither XML nor JSON handle binary data well. To store binary data in either format, you must first convert it to Base64, which introduces an enormous amount of storage and transmission overhead.</p>
<p>Canonical S-Expressions offers none of the overhead of XML and doesn't claim to do type conversions. Since you'll need a reader anyway, you can do your type conversions in that step.</p>
<h3>Further Thoughts and Alternatives</h3>
<p>In practice, having some type data assistance does offer benefits. It makes your reader simpler, and it makes the format more pleasant to work with, and so while I appreciate cxesp's simplicity, I find working with it to be more challenging than it should be.</p>
<p>One thought that I keep having while I'm using csexp is to use the type hints to store information such as the data type. Imagine if instead of:</p>
<div class="highlight"><pre><span></span><code><span class="mi">20</span><span class="o">:</span><span class="mi">2019</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="mi">02</span><span class="n">T07</span><span class="o">:</span><span class="mi">11</span><span class="o">:</span><span class="mi">07</span><span class="n">Z</span>
</code></pre></div>
<p>We instead stored:</p>
<div class="highlight"><pre><span></span><code><span class="o">[</span><span class="n">iso8601</span><span class="o">]</span><span class="mi">20</span><span class="err">:</span><span class="mi">2019</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="mi">02</span><span class="nl">T07</span><span class="p">:</span><span class="mi">11</span><span class="err">:</span><span class="mi">07</span><span class="n">Z</span>
</code></pre></div>
<p>That would give us the data type and we could let the reader take some of the work off of our programming logic. This is similar to JSON-LD's method of storing semantic data.</p>
<p>I personally like this idea, but it requires changes to the readers to recognize a new "Semantic Canonical S-Expression".</p>
<p>A simpler idea would be to store some type information alongside the data, so instead of 3:253, you might store I3:253, with "I" indicating that the value is an integer. This is exactly what the Bencoding format does. Bencoding offers many of the same benefits of CSEXP, but because it also supports types, is a bit easier to work with. The downside, as always, is that this helpfulness comes at the cost of storage and bandwidth.</p>
<p>Other formats exist as well. I previously mentioned Bencoding, but there is also MessagePack, ASN.1, CBOR, and the newest, Preserves. Each of these has a different approach, though they center around the same problem- making it easy to store arbitrary data, especially binary data, on disk and on the network.</p>
<p>It's beyond the scope of this post to delve into each of them. I think Preserves is the most interesting of the formats. It's certainly the most expressive despite being compact, but since I haven't used it I don't know if that expressiveness will be something I need or if I could simply use Bencoding or MessagePack to the same effect.</p>
<h3>Conclusion</h3>
<p>Canonical S-Expressions are a great, flexible, compact data format. It's very fast and efficient. If you have straightforward needs, it's certainly worth checking out. In my use case, Datashards, it fits our current needs. If we end up wanting to store more complex data structures in the format, such as associative arrays, that will be the time to re-evaluate the format choice to see if something else would be a better fit.</p>Is the AGPL Broken?2019-03-25T00:00:00-04:002019-03-25T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2019-03-25:/blog/2019/03/25/is-the-agpl-broken/<h3>Introduction</h3>
<p>Just over a year ago, <a href="https://dustycloud.org">Chris Webber</a> gave a talk at CopyleftConf about how the AGPL is incompatible with a style of computing.</p>
<p>If you want to read the slides, they're at: https://dustycloud.org/misc/boundaries-on-network-copyleft.pdf</p>
<p>Sadly there hasn't been much discussion about it since, so I'm …</p><h3>Introduction</h3>
<p>Just over a year ago, <a href="https://dustycloud.org">Chris Webber</a> gave a talk at CopyleftConf about how the AGPL is incompatible with a style of computing.</p>
<p>If you want to read the slides, they're at: https://dustycloud.org/misc/boundaries-on-network-copyleft.pdf</p>
<p>Sadly there hasn't been much discussion about it since, so I'm going to throw my hat into this rodeo- or some metaphor to that effect.</p>
<p>Before we wrestle with bulls, let's talk about the goal of the <a href="https://www.gnu.org/licenses/agpl-3.0.en.html">AGPL</a> and why it's important in the Free Software ecosystem.</p>
<p>As most people reading this probably already know, the <a href="https://www.gnu.org/licenses/gpl-3.0.en.html">GNU GPL</a> is a license that says that if you have a program, you're entitled to use it, copy it and modify it and that if you distribute it to others, you must do so under the same terms that you received it. It's "Share and Share Alike"</p>
<p>But what does this mean when we have applications that run remotely, such as web applications where executing the program means executing code on someone else's computer? The AGPL states that if you release a program under the AGPL and make it available to others that they have the same obligation to release it to others, whether you release the program as a binary or make it accessible for execution over a network.</p>
<p>This is a good thing in my opinion. Running a program in a networked way to get around the GPL is an anti-social thing to do.</p>
<p>With that out of the way, let's dive in.</p>
<h3>A simple program</h3>
<p>Let's first begin with the idea of a program where state is captured inside execution, rather than in variables. If you know what a closure is, then you can skim or skip this part.</p>
<p>If you don't know what a closure is, you might be wondering what the heck I'm talking about, but it's really not that hard to imagine. Let's take an example from Chris's <a href="https://dustycloud.org/misc/goblins/tutorial.html#%28part._.Vats__actors__spawning__and_immediate_calls%29">own work</a></p>
<p>Chris wrote their code in Scheme. I think the use of a Lisp can lead people to come to the conclusion that this is somehow a Lisp related issue, so I'm going to write my code in Python in order to show that the issue is universal.</p>
<p>Chris proposes that some programs may contain private data but at the same time be stateless. This was hard for me to wrap my head around at first, but we can write a program like this fairly easily:</p>
<div class="highlight"><pre><span></span><code> def make_greeter(greeter_name):
return lambda guest_name: print(f"Hi {guest_name}, I'm {greeter_name}!")
</code></pre></div>
<p>With this, we can construct a greeter named Alice</p>
<div class="highlight"><pre><span></span><code> alice = make_greeter("Alice")
alice("Bob)
</code></pre></div>
<p>And we'd get back "Hi Bob, I'm Alice". What's important here is that the alice function doesn't maintain state. The "Aliceness" is constructed at the time the function is defined.</p>
<p>The data in this case is actually the "Bob" string and not the "Alice" string. The "Alice" string is part of the <code>alice</code> function's executable code.</p>
<p>It's a nifty trick, but it has some deeper implications.</p>
<h3>Turning our program into a service</h3>
<p>Imagine that instead of being generated on the Python shell, there was some external database, and instead of just being a name, the function also contained private information. </p>
<p>Let's rewrite our program with that in mind. We'll create a database of people and their favorite colors.</p>
<div class="highlight"><pre><span></span><code> db = {
'alice': 'red',
'bob': 'blue'}
def make_person(name, color):
return lambda guest_name: print(f"Hi {guesprogramming model.t_name}, I'm {name} and I like {color}")
people = [make_person(*record) for record in db.items()]
</code></pre></div>
<p>Remember, our secrets aren't contained within our database- they're contained within the functions themselves. While this example is trivial, we're starting to see how this could become interesting.</p>
<p>Let's up the ante a bit by turning this into a network application.</p>
<div class="highlight"><pre><span></span><code> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">abort</span><span class="p">,</span> <span class="n">request</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
<span class="n">db</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'Alice'</span><span class="p">:</span> <span class="s1">'red'</span><span class="p">,</span>
<span class="s1">'Bob'</span><span class="p">:</span> <span class="s1">'blue'</span><span class="p">}</span>
<span class="k">def</span> <span class="nf">make_person</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">color</span><span class="p">):</span>
<span class="k">return</span> <span class="k">lambda</span> <span class="n">guest_name</span><span class="p">:</span> <span class="sa">f</span><span class="s2">"Hi </span><span class="si">{</span><span class="n">guest_name</span><span class="si">}</span><span class="s2">, I'm </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2"> and I like </span><span class="si">{</span><span class="n">color</span><span class="si">}</span><span class="s2">.</span><span class="se">\n</span><span class="s2">"</span>
<span class="n">people</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">make_person</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">color</span><span class="p">)</span> \
<span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">color</span><span class="p">)</span> <span class="ow">in</span> <span class="n">db</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/<person>'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">show_greeting</span><span class="p">(</span><span class="n">person</span><span class="p">):</span>
<span class="n">guest</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'guest'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">people</span><span class="p">[</span><span class="n">person</span><span class="p">](</span><span class="n">guest</span><span class="p">)</span>
<span class="n">abort</span><span class="p">(</span><span class="mi">404</span><span class="p">)</span>
</code></pre></div>
<p>And run it:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="n">serge</span><span class="nv">@laptop</span><span class="err">:</span><span class="o">~</span><span class="err">$</span><span class="w"> </span><span class="n">curl</span><span class="w"> </span><span class="nl">http</span><span class="p">:</span><span class="o">//</span><span class="nl">localhost</span><span class="p">:</span><span class="mi">5000</span><span class="o">/</span><span class="n">Alice</span><span class="vm">?</span><span class="n">guest</span><span class="o">=</span><span class="n">Bob</span>
<span class="w"> </span><span class="n">Hi</span><span class="w"> </span><span class="n">Bob</span><span class="p">,</span><span class="w"> </span><span class="n">I</span><span class="err">'</span><span class="n">m</span><span class="w"> </span><span class="n">Alice</span><span class="w"> </span><span class="ow">and</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="ow">like</span><span class="w"> </span><span class="n">red</span><span class="p">.</span>
</code></pre></div>
<p>Nifty, but not especially different from the previous example, except as it applies to the AGPL.</p>
<p>We can take this example in one of two directions, both of which I believe breaks the AGPL.</p>
<p>The first is that we might imagine the database contains some other secrets, but that we're encoding these secrets as code. Let's imagine that we have a service that lets doctors and other services that we explicitly permit to have access to health-related data about us.</p>
<p>As privacy-oriented developers, we may want to self-host this application. I certainly feel better about running my own services, especially where sensitive/private data is concerned.</p>
<p>As far as the standard GPL is concerned, this is no problem. My private version of my application that only runs on my computer is entirely mine. But the AGPL is different- the network accessibility of the service places the program under the same distribution terms as we would have if we were to distribute the program.</p>
<h3>Configuration as Code</h3>
<p>How realistic is this scenario of using code for configuration? It's far more common than you might originally think. As Chris's talk points out, it's extremely common in Lisp to use this method- but it's not limited to Lisp by any means. Several popular Python web frameworks use a <code>config.py</code> file, and PHP developers use <code>config.php</code>.</p>
<p>This is because while the licenses do not pertain to running environments, these configuration systems turn the configuration "data" into running an executable. That is distinct from, for example, pulling data from a YAML or <code>config.ini</code> file because in a <code>config.py</code> file, the file is being interpreted <strong>as code</strong> and becoming part of the program itself.</p>
<p>This is largely a non-issue because in a vast majority of cases there is a distinction between the types of static variables placed inside a configuration file and the dynamic code that's inside the program files, but this doesn't have to be the case. It's possible to write configuration that contains executable code, and if that executable code modifies the behavior of the application itself, then it is indistinguishable from program code. </p>
<p>Does this mean you can't write a Python application that uses <code>config.py</code> or a PHP program that uses <code>config.php</code> under the AGPL? In most cases, the difference between simply storing a variable statically inside one file or another would not make a difference, but as the complexity of configuration may grow to include functionality, that line begins to blur, and while I'm not a lawyer, I believe that <em>without relicensing the configuration files</em>, the answer is that if your configuration is sufficiently complex that it is indistinguishable from code that you will need to publish it as code under the AGPL.</p>
<p>Obviously <em>this is not the intent of the AGPL</em>, and this specific scenario is easily remedied by separating out and separately licensing the config files, but this is a conscious action that the developer must take.</p>
<h3>Plugins</h3>
<p>Let's take on a more complex version of this problem: What happens when applications are not simply monolithic, stand-alone things, but when they include components that are external in some way?</p>
<p>Chris in a <a href="https://www.reddit.com/r/freesoftware/comments/fop3bs/breaking_the_agpl/flgpvhu/">reddit reply</a> to this post, mentions browsers- so let's use that as an example. If you're reading this, you most likely are doing so on a web browser. You're also likely to have one or more plugins. Plugins are application logic that extends the functionality of your application in some way. The plugins may be under a variety of licenses- anything from extremely permissive to entirely proprietary.</p>
<p>If your browser is under the GPL, the waters become very murky as it relates to the licensing requirements of plugins. Wordpress, the popular CMS and blogging platform, has <a href="https://kinsta.com/learn/wordpress-gpl/">stated</a> that Wordpress plugins should, (or possibly must) be released under the GPL. That is because a plugin is not a stand-alone work. A plugin depends on the Wordpress application framework, and thus plugins are derived (or as GPLv3 calls it, "based on") the original program.</p>
<p>For GNU GPL applications, this is a bit of an oddity, as while Wordpress may require plugins be under the GPL, they cannot compel users running proprietary plugins to provide source code to them. With the AGPL, a network user of the program has the same rights as person downloading the program.</p>
<p>This is a lot to take in, but we're not quite done yet. In Spritely Goblins, the system Chris is developing, there is no distinction between a local program execution and one that runs on the network. While some developers may be used to thinking about remote procedure calls and remote APIs, the Goblins model makes this distinction largely invisible to the user and even the developer- program logic may be run locally, on a nearby server owned by the same person, or halfway around the world by someone, they've never met.</p>
<p>Goblins, by design, erases the distinction for a programmer about whether the code being run is internally or externally. It erases the distinction for a programmer about whether or not the code is being run <a href="https://www.gnu.org/licenses/gpl-faq.en.html#GPLInProprietarySystem">at arm's length</a>.</p>
<p>Under the GPL, this is no problem- network services are at arm's length and thus there's no problem with integrating your GPLed internal code with some external proprietary service. But under the AGPL, network services are explicitly included.</p>
<h3>A brief review</h3>
<p>...That was a lot to cover, so let's review briefly.</p>
<ul>
<li>
<p>Some programs are going to be Free Software, but contain "proprietary parts" because they need to for privacy reasons.</p>
</li>
<li>
<p>Plugins that are written for an AGPLed system must be AGPLed, even if they operate across the network</p>
</li>
<li>
<p>Therefore we have an impedance mismatch between the intent of the AGPL (to protect Software Freedom) and personal privacy, which is amplified on a system that makes no distinction between local and network code</p>
</li>
</ul>
<h3>In the land of tomorrow...</h3>
<p>Now that this is covered, let's get weird...</p>
<p>Spritely Goblins has the potential to do more than just provide remote procedure calls for remote applications- it's designed so that it could also take object code and safely execute it locally.</p>
<p>This may seem strange at first, but a longer-term goal of Spritely appears to be to take in-memory object code and ship it to another machine where it can be safely executed. I use the adjective "apparently" here because I don't see mention of this in the Spritely docs, but it is something Chris and I have discussed privately.</p>
<p>In terms of functionality, this is extremely powerful, but it gets complicated when we talk about source code requirements. As people who have done work in the field of Reproducible Builds know, making software reproducible is not trivial, and if instead of shipping object code, we had to ship source code around, this would be a large burden on the recipient system to then need to not only build the source but possibly also to replicate the remote environment.</p>The GNU Kind Communication Guidelines are Bad for Free Software2018-10-19T00:00:00-04:002018-10-19T00:00:00-04:00Serge Wroclawskitag:blog.emacsen.net,2018-10-19:/blog/2018/10/19/gkcg-is-bad/<p>I have been a long time supporter of the Free Software movement and the Free Software Foundation. I care deeply about Free Software, and it's the ideals of the movement that have been at the cornernerstone of much of what I've spent the last 20 years of my life using …</p><p>I have been a long time supporter of the Free Software movement and the Free Software Foundation. I care deeply about Free Software, and it's the ideals of the movement that have been at the cornernerstone of much of what I've spent the last 20 years of my life using and promoting. Last week's announcement[1] of the GNU Kind Communication Guidelines[2] were a grave dissapointmnet.</p>
<p>I came to Free Software in late 1997, just as the Open Source movement was beginning. I saw the change as people started to take sides, disassociating themselves from the social/political movement of Free Software and onto the business-friendly, apolitical side of Open Source.[3] I choose then and remain now a Free Software supporter. I support Free Software as a movement because I want to see a better world and see Free Software and Free Culture as part of those values.</p>
<p>Free Software's appeal to me was and continues to be its practical ethics and unapologetic political stance. While Free Software began officially in the 1980s, the movement itself is more relevant than every. Today most of us interact with dozens of computers a day- in our phones, in our cars, televisions, even our refrigerators and thermostats. And while it's possible to run a 100% Free Software desktop, most of these devices are outside of our control or even understanding. Free Software's goal of self-determination is more important than ever.</p>
<p>Richard Stallman (RMS), the father of Free Software, said that software could not be labeled as Free Software unless the freedom it provides are the same for everyone.[4] I have always held this value to be true, not just as an abtract concept but as a core value. Our goal is not to simply to provide liberty to those in the upper eschelons of society, who have won the genetic lottery of being born into wealth or privledge, but to provide those freedom to everyone, and offer everyone the opportunity to realize their potential.</p>
<p>Free Software has historically put this idea into practice. The One Laptop Per Child project, whose goal it was to bring technology to impoverished children all over the world, has explicitly stated its support for Free Software[5]. With computing becoming increasingly inexpensive, it is now more possible than ever to give access to everyone, no matter where in the world they live.</p>
<p>Universal access is not a new idea in Free Software. At the beginning of the GNU project, Unix machines were expensive and beyond the reach of most people. With the advent of Linux and *BSD, running a Free Software operating system was now possible for people to have at home. Even before GNU, Stallman rebelled against a computer "aristocracy", believing that everyone should have access to computers.[6]</p>
<p>The idea of universal access and equality is baked into the history and fabric of the Free Software movement. When we violate these ideals, we harm the movement as a whole. If we can't put our ethical principles into effect in the way we do through coding and licensing, then our goals are not fully realized.</p>
<p>In the same way that the freedom of Free Software leads to practical benefits[7], bringing in people from different backgrounds and experiences also has practical benefits such as increased creativity and better work overall[8][9]. Ensuring that our projects are not only open to but openly inviting people from all walks of life is vital to the long-term survival of our movement.</p>
<p>In the GNU Kind Communication Guidelines announcement, RMS explicitly states that diversity is not a goal of his. This is a mistake both for him personally but also in his role as a leader of the Free Software movement, and the GNU project. GNU is more than a series of software programs, it is the realization of a Utopian vision. Software that is Free only for some is not truly Free. In ignoring the larger social constructs about why someone might feel excluded, he is turning a blind eye to the people who would gain the most from the benefits that Free Software provides, especially in the areas of surveillance and access. These are the same issues that RMS explicitly touts as reasons to use Free Software. [10][11][12][13]</p>
<p>In a post I made in 2012, I described my experience of accidentally going to a women's only Python tutorial and seeing how the innocuous behavior of men in the class changed the dynamics of the women participants. The behaviors that were being exhibited were not exclusionary or aggressive, but they had the effect of women in the class tuning out.[14] From this experience and subsequent discussions with women and other marginalized groups, I've come to understand that being inclusive requires more than just the absence of barriers- it requires being openly welcoming and even changing one's approach to fit the cultural norms of the group one is trying to attract. As the Python tutorial showed me, behavior that would have been entirely normal in another context had the effect of turning people away from the material, and by extension away from programming. That experience helped me see why the onus on communication must be on the speaker to understand their audience.</p>
<p>The GNU Kind Communication Guidelines takes the opposite approach and begins with addressing the recipient of a message, instructing them to hear what is said to them in good faith. Talking to the recipient first, before the speaker starts the conversation off wrongly,places the bulk of the emotional labor on the recipient of a message rather than on the sender. It's precisely this attitude that has gotten us where we are, and what needs to change, not just to attract women to our community, but to make the development of Free Software something people enjoy doing. </p>
<p>The goal of Codes of Conduct is to spell out what people who join a community can expect from that community. It's a sign that these issues are taken seriously and that someone who joins can and should expect to be treated with respect by existing members.[15] The GNU Kind Communication is not wrong in its words, it's wrong in that it only addresses a tiny slither of the problems we have in the community and turns a blind eye to the larger social contexts around why Free Software remains as insular as it does.</p>
<p>Stallman states in his announcement that his concern is in keeping the developers of a project happy and productive. This is a complex topic, especially as so many individuals in Free Software (including myself) have felt at time ostracized by others for being "different". We have found our home amongst others like ourselves and want that home to continue. In addition, as I saw when I researched this issue during my time in OpenStreetMap, a small percentage of a project's contributors do a majority of the work. On a cursory look, it would appear that the best thing to do would be to optimize solely for the minority of developers doing the bulk of the work. </p>
<p>The problem with this argument is that it ignores both the larger social context of Free Software and the problem of the invisible developer. The ultimate goal of the Free Software movement is not just to build better software, but to build a better world. We can't build a better world when groups of people feel excluded. And if people feel excluded, they will never join the community and thus we'll never see their contributions.</p>
<p>In a recent article, RMS reaffirmed that Open Source is "amoral" and "depoliticized".[16] I agree. Free Software has been the moral choice for those of us who want to do more than write software, we want to make the world a better place. But that can't happen if it's not better for all, no matter their gender, their race, sexual orientation, geography, etc. We want Freedom in our Free Software, for everyone.</p>
<p>The GNU Kind Communication Guidelines are a mistake for Free Software and I urge those who care about Freedom as a principle to ignore it and adopt more comprehensive Codes of Conduct which provide guidelines on what behavior is acceptable, what behavior is not, and what steps the community will do in enforcing them. I also urge the GNU project as a whole to reject these guidelines, as well as RMS to redact or entirely rewrite the guidelines and provide the kind of comprehensive changes that have been talked about in this and other critiques.</p>
<p>[1] http://lists.gnu.org/archive/html/info-gnu/2018-10/msg00001.html
[2] https://www.gnu.org/philosophy/kind-communication.html
[3] https://www.gnu.org/philosophy/open-source-misses-the-point.html
[4] https://www.gnu.org/philosophy/free-sw.en.html
[5] http://one.laptop.org/about/5-core-principles
[6] https://www.oreilly.com/openbook/freedom/ch07.html
[7] https://www.edutopia.org/blog/benefits-free-software-shahzad-saeed
[8] https://www.npr.org/2018/07/02/625426015/the-edge-effect
[9] https://www.scientificamerican.com/article/how-diversity-makes-us-smarter/
[10] https://www.wired.com/2013/10/a-necessary-evil-what-it-takes-for-democracy-to-survive-surveillance/
[11] https://www.gnu.org/philosophy/right-to-read.html
[12] https://www.gnu.org/proprietary/malware-microsoft.html
[13] https://stallman.org/apple.html
[14] https://blog.emacsen.net/blog/2012/06/07/observations-from-a-python-workshop/
[15] https://opensource.guide/code-of-conduct/
[16] https://newleftreview.org/II/113/richard-stallman-talking-to-the-mailman</p>Bridging Jitsi Meeting with Twilio Using Python2018-02-16T00:00:00-05:002018-02-16T00:00:00-05:00Serge Wroclawskitag:blog.emacsen.net,2018-02-16:/blog/2018/02/16/jitsi-meet-and-twilio-with-python/<p>Jitsi Meet is a Free Software audio and video conferencing platform
that allows for people around the world to participate in a video
conference without proprietary software like Zoom or Google Meet.</p>
<p>Jitsi has an add-on program called Jigasi that allows for call-ins
(and even call-outs). Unfortunately, while Jitsi Meet …</p><p>Jitsi Meet is a Free Software audio and video conferencing platform
that allows for people around the world to participate in a video
conference without proprietary software like Zoom or Google Meet.</p>
<p>Jitsi has an add-on program called Jigasi that allows for call-ins
(and even call-outs). Unfortunately, while Jitsi Meet is well
documented, Jigagi has less documentation. In this guide, I will
demonstrate how to set up Jitsi Meet and Jigasi using the Twilio phone
platform. </p>
<p>This post will try to cover the basics of the various components, but
I am not an expert on any of them- I just managed to get everything
working after a lot of trial and error.</p>
<h1>Connecting to the Phone Network</h1>
<p>Jitsi is great for computer based meetings. It even has an iOS and
Android app, but occassionally we need to support phone dial-in
attendees. Jitsi uses a media transport called WebRTC, while VOIP
software most commonly uses a protocol called SIP.</p>
<p>This means we need to bridge both the technical protocols but also the
very different way that these two protocols see the world.</p>
<p>Traditionally, making a voice-enabled application would involve
setting up a PBX. PBX stands for Private Branch Exchange, which is
another way to say that a PBX system works like a small phone company.</p>
<p>In the past, PBX systems were proprietary and expensive, but Asterisk
changed all that. Asterisk and other SIP FLOSS servers can run on
relatively small installations, but still require a good deal of
specialized knowledge to use. In addition you will still need a "trunk
provider" to connect your installation to the phone network.</p>
<p>Twilio is a phone provider that makes it easy for programmers to build
phone applications by simply putting up a web server. It requires no
proprietary software on the client end, easy sign-up and competitive prices.</p>
<p>The largest downside of Twilio is that because of it's specialized
API, there is a bit of vender lock-in, unlike using a plain SIP trunk
provider and connecting it to a program like Asterisk or
FreeSwitch. On the upside, the Twilio API is very somple and its tools
make debugging applications a breeeze.</p>
<p>Since we only want one or two numbers and an easy installation, we're
going to go ahead and use Twilio for this application.</p>
<h1>Another Web Server?</h1>
<p>Twilio has an event driven API. When a telephonony event occurs,
Twilio triggers an event on its end. One option for events is to hit a
specified HTTP endpoint. We can run own webserver and direct Twilio on
what to do next.</p>
<p>For this particular application, I'm going to use the popular Python
Flask web framework because it's easy and because Twilio offers an SDK
that makes using it very easy, but you could use any web server you
like.</p>
<h1>Installing Jitsi Meet</h1>
<p><em>If you already have Jitsi meet installed, you can feel free to skip
this section.</em></p>
<p>Jitsi Meet itself is fairly well documented. To make deployment
easier, I've been using the official Jitsi Meet Docker image. The
installation manual for the Docker install is available <a href="https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-docker">here</a>.</p>
<p>While not strictly necessary, since you will need to run additional
services anyway, I'm using a SSL reverse proxy that integrates Let's
Encrypt called <a href="https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion">docker-compose-letsencrypt-nginx-proxy-companion</a>.</p>
<p>If you want to do the same , you will need to set your
<code>LETSENCRYPT_DOMAIN</code> and <code>LETSENCRYPT_EMAIL</code> in your Jitsi <code>.env</code>, but
don't set <code>ENABLE_LETSENCRYPT</code>. In addition, you will need to set
<code>DISABLE_HTTPS</code>.</p>
<p>It should be mentioned that SSL is mandatory for WebRTC on the browser
level, so using <strong>some</strong> SSL configuration is necessary, whether it's
through a proxy or Jitsi itself.</p>
<p>You'll also need to change your <code>docker-compose.yml</code> file. Add
<code>VIRTUAL_HOST=${LETSENCRYPT_DOMAIN}</code> and
<code>LETSENCRYPT_HOST=${LETSENCRYPT_DOMAIN}</code> to your <code>web</code> section
<code>environment</code> section. You'll also need to add the proxy network
(which defaults to <code>webproxy</code> to <code>web</code>'s networks. Just add
<code>webproxy:</code> there and in the networks section add:</p>
<div class="highlight"><pre><span></span><code> webproxy:
external:
name: webproxy
</code></pre></div>
<p>If you're already familiar with this proxy companion, or
<a href="https://hub.docker.com/r/jwilder/nginx-proxy">jwilder/nginx-proxy</a> then this will be familiar to you.</p>
<p>Once that's working to your satisfaction, let's move onto the next
step.</p>
<h1>Dial-in Number</h1>
<p>The next step is to sign up for Twilio and get a phone number. This is
the number that people will use when they dial into the phone
conference.</p>
<p>Because this phone connection connects to the standard phone system,
you will need to pay or this, but the prices are relatively
inexpensive. In my experience, my Twilio costs were about $3 a month
for light/moderate usage.</p>
<p>As an aside, it should be mentioned that Twilio also offers their own
WebRTC-based videoconferencing system. If we only cared about pricing,
then it would be a safe win to use thier system, but we are using
Jitsi because we also care about the Software Freedom.</p>
<h1>Twilio's SIP</h1>
<p>In addition to the number, you will also need to set up a SIP
domain. Twilio offers a number of SIP offerings and navigating the
system is a little confusing. I found <a href="https://www.twilio.com/blog/registering-sip-phone-twilio-inbound-outbound">this</a> article on sip phones from
Twilio was very informative. </p>
<p>You will need a SIP domain that represents your organization, but
since you can also have multiple SIP domains, that is up to
you. Similarly, you can choose a username independently of anything
else, though <a href="https://www.twilio.com/blog/registering-sip-phone-twilio-inbound-outbound">this</a> blog post from Twilio suggests using an E.164 format
phone number for the username.</p>
<p>You'll also need to set parameters around network address based logins
and other settings. The Twilio documentation mentions being able to
create this configuration through a RESTful Interface, but since this
is a one-off, I think using the GUI is easiest.</p>
<p>At this point I'll assume both your Twilio and SIP configuration are
working and you're able to register</p>
<h1>Web Server Setup and CORS</h1>
<p>Setting up a web server is outside the scope of this tutorial. I'm
going to assume you've already set up a web service before and or have
an understanding of HTTP methods (GET, POST, etc.) as well as the
basics of XML and JSON encoding.</p>
<p>You'll need to stand up a web server somewhere. Since you already have
a Jitsi instance, standing up another service should be fairly easy
for you. If you want to use Docker, look at the Dockerfile included in
the project for an example of setting up a Docker instance.</p>
<p>You may also find <a href="https://ngrok.com/">ngrok</a> to be a nice tool to use during testing, but
this is optional.</p>
<p>For Twilio's purposes, we just need a web server running somewhere it
can talk to, but we'll also be setting up a could of HTTP endpoints
for Jitsi Meet clients, and because of that, we'll need to set up
<a href="https://en.wikipedia.org/wiki/Cross-origin_resource_sharing">Cross-Origin Resource Sharing</a> or CORS on the web server. In my example
we're going to configure the server to return the header:</p>
<p><code>Access-Control-Allow-Origin: *</code></p>
<p>But you may want to restrict the respone only to your Jiti
domain but I have't bothered in my example.</p>
<p>You will also need to configure SSL access to your web server- not for
Twilio, but later on when we configure Jitsi Meet. You can certainly
wait on this step until you're ready to test the Jitsi Meet
components, but it's good to know that you'll need this eventually.</p>
<p>For the remainder of this tutorial, I'm going to assume your web
server is up and running and at <code>https://example.com</code>.</p>
<h1>Configuring Twilio to Answer a Call</h1>
<p>/In this section we'll talk about Twilio and using it's event-driven
system to answer calls. If you're already familiar with Twilio, you
can safely skip this section./</p>
<p>In the old days, setting up a PBX was expensive and required both
proprietary hardware and software. That all changed when <a href="https://www.asterisk.org/">Asterisk</a> came
out. With Asterisk, you could run your own PBX, either physically or
virtually, with Free Software. You just had to learn a little
telephony nomenclature and you could set up your own virtual PBX in an
afternoon. Today, several options exist for running your own PBX,
including FreeSWITCH, OpenSIPS and FreePBX (which sits on top of
Asterisk). All of these systems are wonderful, but even in the best
case, require some understanding of telephony and creation of a
<em>dialplan</em>. A dialplan is nothing more than a set of instructions that
are carried out when a telephony event occurs. For example your office
might want the CEO's phone to ring in their office, but also ring
their administrative assistant, or you might want everyone in the
office to have their own extension, but only certain extensions have
direct dial in numbers that people can call externally.</p>
<p>Twilio abstracts the dialplan idea into a series of events that you
configure it to respond to. You can choose how to respond to those
events, but in our case, we will use webhooks, which are nothing more
than simple HTTP endpoints.</p>
<p>Our first example will be to configure our phone number to say a
greeting and then hang up. We could easily configure this with a
static file, even on the Twilio website, but by testing it on our web
server, we're also ensuring that our web server is configured
properly.</p>
<p>Twilio provides an SDK that abstracts its domain specific XML, TwiML
and makes it easy to use. You don't need the SDK, of course. You could
do it all yourself manually. </p>
<p>I'm going to name the HTTP endpoint "answer", since that is the event
that we'll respond to. I'll also be setting up some basic Flask
things. If you've worked with Flask before or most web frameworks,
nothing here will be especially new.</p>
<div class="highlight"><pre><span></span><code> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>
<span class="kn">from</span> <span class="nn">twilio.twiml.voice_response</span> <span class="kn">import</span> <span class="n">VoiceResponse</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/answer"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">answer</span><span class="p">():</span>
<span class="w"> </span><span class="sd">"""Respond to incoming phone calls with a greeting"""</span>
<span class="n">resp</span> <span class="o">=</span> <span class="n">VoiceResponse</span><span class="p">()</span>
<span class="n">resp</span><span class="o">.</span><span class="n">say</span><span class="p">(</span><span class="s2">"Hello and welcome to the conferencing system"</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">str</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">host</span><span class="o">=</span><span class="s1">'0.0.0.0'</span><span class="p">,</span> <span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>
<p>A majority of that program is just setting up the web server, but we
can see just how easy it is to set up.</p>
<p>If you look at the result of hitting that endpoint, you will see
something that looks like</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="w"> </span><span class="nt"><Response></span>
<span class="w"> </span><span class="nt"><Say></span>Hello<span class="w"> </span>and<span class="w"> </span>welcome<span class="w"> </span>to<span class="w"> </span>the<span class="w"> </span>conferencing<span class="w"> </span>system<span class="nt"></Say></span>
<span class="w"> </span><span class="nt"></Response></span>
</code></pre></div>
<p>I've formatted the output, but you can see that the result is a small
XML document. We could just store that as a static file, but we're
going to need to make our site more interactive later.</p>
<h1>Checking our SIP Configuration</h1>
<p><em>If you've already configured and tested your SIP endpoint, this step
is unnecessary</em></p>
<p>With our telephone number and web server configured, let's turn our
attention back to our SIP configuration. If you haven't done that
already, go to <a href="https://www.twilio.com/console/voice/sip/endpoints/">Programmable SIP Domains</a> and add a new domain for
yourself.</p>
<p>Then go ahead and add a user for that domain. As mentioned earlier,
one practice is to name the user the same as the phone number, but
that's entirely optional.</p>
<p>What's not optional is making the SIP domain, user for that domain and
also setting the IP address ranges that will connect to the
endpoint. This will be your Jigasi server's IP, but I also recommend
testing the SIP endpoint with a SIP softphone such as Linphone or
Zoiper, so you'll want to add the IP address of the computer you'll be
testing it from as well.</p>
<p>If you haven't used Twilio's SIP before, one small gotcha that I
encountered is that the SIP domain is not always the same as the
server, so I had to add <code>us1</code> to the sip domain, such as
<code>myuser@mydomain.sip.us1.twilio.com</code>.</p>
<p>Just be sure that your SIP phone can connect to the endpoint
successfully. We'll be configuring our number to ring the SIP phone
next. so it's a good time to ensure that this part is working before
we move on.</p>
<h1>Configuring our number to call our SIP Endpoint</h1>
<p>We have our web server and our SIP endpoint both working, so now it's
time to connect them together.</p>
<p>Since we're now dealing with a bunch of configuration, I'm going to
use <a href="https://pypi.org/project/python-dotenv/">dotenv</a> to make it easy for me to store configuration separately
from the application. In production, I'm using Docker, so I'll be
storing my configuration there instead, but this is a nice bridge
between the two. We'll then use <code>environ</code> to retrieve our
configuration.</p>
<p>Let's store our SIP user with domain as SIP<sub>URI</sub>.</p>
<p>Then when someone calls our number, we'll have it call the SIP
endpoint. When that happens, your softphone should ring and you'll be
able to talk to yourself.</p>
<div class="highlight"><pre><span></span><code> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>
<span class="kn">from</span> <span class="nn">twilio.twiml.voice_response</span> <span class="kn">import</span> <span class="n">VoiceResponse</span><span class="p">,</span> <span class="n">Dial</span>
<span class="kn">from</span> <span class="nn">dotenv</span> <span class="kn">import</span> <span class="n">load_dotenv</span>
<span class="kn">from</span> <span class="nn">os</span> <span class="kn">import</span> <span class="n">environ</span>
<span class="n">load_dotenv</span><span class="p">()</span>
<span class="n">SIP_URI</span> <span class="o">=</span> <span class="n">environ</span><span class="p">[</span><span class="s1">'SIP_URI'</span><span class="p">]</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/answer"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">answer</span><span class="p">():</span>
<span class="w"> </span><span class="sd">"""Call the SIP endpoint"""</span>
<span class="n">resp</span> <span class="o">=</span> <span class="n">VoiceResponse</span><span class="p">()</span>
<span class="n">dial</span> <span class="o">=</span> <span class="n">Dial</span><span class="p">()</span>
<span class="n">dial</span><span class="o">.</span><span class="n">sip</span><span class="p">(</span><span class="sa">f</span><span class="s2">"sip:</span><span class="si">{</span><span class="n">SIP_URI</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="n">resp</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">dial</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">str</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">host</span><span class="o">=</span><span class="s1">'0.0.0.0'</span><span class="p">,</span> <span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>
<p>Not a lot of change here, but now when we call our phone number, it
should call our SIP user, which is connected to the softphone.</p>
<p>If this all works, you're cooking with gas and it's time to move on to
configuring Jigasi itself!</p>
<h1>Configuring Jigasi</h1>
<p>In this example, I'm going to be using the Docker installation of
Jitsi. In this configuration, a lot of the details have been
abstracted away and only need to be set inside the <code>.env</code> file your
Jitsi installation uses.</p>
<p>If you're not using the Docker installation, you'll need to make the
changes in the config files themselves.</p>
<p>Here's the relevant part of the Jitsi <code>.env</code></p>
<div class="highlight"><pre><span></span><code> #
# Basic Jigasi configuration options (needed for SIP gateway support)
#
# SIP URI for incoming / outgoing calls
JIGASI_SIP_URI=SIP_USER
# Password for the specified SIP account as a clear text
JIGASI_SIP_PASSWORD=MY_SIP_PASSWORD
# SIP server (use the SIP account domain if in doubt)
JIGASI_SIP_SERVER=MYSIP_DOMAIN
# SIP server port
JIGASI_SIP_PORT=5060
# SIP server transport
JIGASI_SIP_TRANSPORT=UDP
</code></pre></div>
<p><code>JIGASI_SIP_URI</code> should be the same as the <code>SIP_URI</code> we set for our
Flask application, <code>JIGASI_SIP_PASSWORD</code> is the password, and
<code>JIGASI_SIP_SERVER</code> should be the SIP Domain, including the <code>us1</code>
part.</p>
<p>Once you do this, you'll need to recreate the Jitsi and Jigasi config
files. If you're using the Docker images, the <code>.env</code> file specifies a
<code>CONFIG</code> variable which stores the location of the configuration
directory.</p>
<p>You'll need to erase that directory and recreate it with:</p>
<div class="highlight"><pre><span></span><code> mkdir -p CONFIG_DIR/{web/letsencrypt,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
</code></pre></div>
<p>Substituing CONFIG<sub>DIR</sub> with the location in <code>CONFIG</code>.</p>
<p>Also you'll need to be sure that from now on, you reference both the
<code>docker-compose.yml</code> as well as the <code>jigasi.yml</code> files, such as:</p>
<div class="highlight"><pre><span></span><code>docker-compose -f docker-compose.yml -f jigasi.yml up -d
</code></pre></div>
<p>Once you make these changes and restart the services, Jigasi should
register as a SIP endpoint (just like the softphone) and be able to
recieve calls. The problem is that it doesn't know which conference to
send the calls to by default.</p>
<p>We can give Jitsi a default conference room for it to use by setting
it in <code>CONFIG/web/config.js</code> as
<code>org.jitsi.jigasi.DEFAULT_JVB_ROOM_NAME</code> but I think a better way is
to modify our Python script to specify the room there.</p>
<p>What we need to do is technically to specify the room name inside of a
SIP header when we make the SIP INVITE. That header is X-Room-Name by
default and we can specify the room name there.</p>
<p>Twilio lets us set SIP headers on the URI, so all we need to do is
specify X-Room-Name on the <code>dial.sip</code> line like so:</p>
<div class="highlight"><pre><span></span><code> dial.sip(f"sip:{SIP_URI}?X-Room-Name=MyDefaultRoomHere")
</code></pre></div>
<p>Now a call to our number will be directed to the <code>MyDefaultRoomHere</code>
room!</p>
<p>Technically we could stop here. If we always know that we want calls
to come into this one room, we don't need to take any further action.</p>
<p>But we probably want features like PIN numbers and other things, so
let's go ahead and add that!</p>
<h1>Mapping PINs to Rooms</h1>
<p>Jitsi Meet has the concept of rooms. Rooms have a unqiue identifier
which we can think of as an access token into the room. We need to map
those room names to digits that we can easily type into the phone.</p>
<p>Then when a caller calls in, we need to ask them for a PIN and then
map that back to a room name, which we then use to tell our python
program where to send them.</p>
<p>This is a bit of a chicken and egg problem, because we need both parts
to fully test this, but I'm going to implement the PIN<->Room Name
mapping first.</p>
<p>We technically could do this entirely in memory, but then if we shut
the program down, we'd lose all the previous mappings, so we need to
serialize this to disk. We could use a full fledged database, but on
my system I only get a few visitors a day on my Jitsi instance and
generate maybe one or two new rooms a week, so a full fledged database
seems like overkill as well, so I'm opting ofr a very simple solution
in the form of a Python library <code>tinydb</code> which works like a
dictionary, but loads the data each time it's called, which means that
while not guaranteed, it's certainly thread safe enough for our this use.</p>
<p>Jitsi Meet makes the calls on the client side, from the web interface,
and this is why we must address the Cross Origin Resource Sharing
issue. Since we're not dealing with any large resource generally,
we'll just put a blanket policy allowing anyone. In production you may
wish to set this to your Jitsi meet URL.</p>
<p>The official Jitsi meet instance server has an instance of mapping the
conference to a PIN at
<code>https://jitsi-api.jitsi.net/conferenceMapper</code>. This URL takes in one
of two parameters through a GET request, either <code>conference</code> or
<code>id</code>. The conference is the full conference name, that is the room
name @ the instance. The ID is what I'm caling the PIN. The result is
a JSON document.</p>
<p>Some tutorials suggest using an auto-incrementing ID, but I think this
is a mistake because even though it doesn't tell you what room you'll
get, it does make it likely that someone could guess the next room
PIN, so instead I'll be using a random number.</p>
<div class="highlight"><pre><span></span><code> <span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">jsonify</span><span class="p">,</span> <span class="n">request</span>
<span class="kn">from</span> <span class="nn">flask_cors</span> <span class="kn">import</span> <span class="n">CORS</span>
<span class="kn">from</span> <span class="nn">tinydb</span> <span class="kn">import</span> <span class="n">TinyDB</span><span class="p">,</span> <span class="n">Query</span>
<span class="kn">from</span> <span class="nn">secrets</span> <span class="kn">import</span> <span class="n">randbelow</span>
<span class="o">...</span>
<span class="n">PIN_DIGITS</span> <span class="o">=</span> <span class="mi">6</span>
<span class="n">DB_FILE</span> <span class="o">=</span> <span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"DB"</span><span class="p">)</span>
<span class="n">db</span> <span class="o">=</span> <span class="n">TinyDB</span><span class="p">(</span><span class="n">DB_FILE</span><span class="p">)</span>
<span class="o">...</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
<span class="n">cors</span> <span class="o">=</span> <span class="n">CORS</span><span class="p">(</span><span class="n">app</span><span class="p">)</span>
<span class="o">...</span>
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/conferenceMapper'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">conference_mapper</span><span class="p">():</span>
<span class="n">pin</span><span class="p">,</span> <span class="n">conference</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'id'</span><span class="p">),</span> <span class="n">request</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'conference'</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">pin</span> <span class="ow">or</span> <span class="n">conference</span><span class="p">:</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s2">"message"</span><span class="p">:</span> <span class="s2">"No conference or id provided"</span><span class="p">,</span>
<span class="s2">"conference"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
<span class="s2">"id"</span><span class="p">:</span> <span class="kc">False</span><span class="p">})</span>
<span class="k">elif</span> <span class="n">pin</span><span class="p">:</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">db</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">Query</span><span class="p">()</span><span class="o">.</span><span class="n">id</span> <span class="o">==</span> <span class="n">pin</span><span class="p">)</span>
<span class="k">if</span> <span class="n">result</span><span class="p">:</span>
<span class="n">conference</span> <span class="o">=</span> <span class="n">result</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">'conference'</span><span class="p">]</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span>
<span class="s2">"message"</span><span class="p">:</span> <span class="s2">"Successfully retrieved conference mapping"</span><span class="p">,</span>
<span class="s2">"id"</span><span class="p">:</span> <span class="n">pin</span><span class="p">,</span>
<span class="s2">"conference"</span><span class="p">:</span> <span class="n">conference</span><span class="p">})</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span>
<span class="s2">"message"</span><span class="p">:</span> <span class="s2">"No conference mapping was found"</span><span class="p">,</span>
<span class="s2">"id"</span><span class="p">:</span> <span class="n">pin</span><span class="p">,</span>
<span class="s2">"conference"</span><span class="p">:</span> <span class="kc">False</span><span class="p">})</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># The conference has been specified- make a new PIN</span>
<span class="n">max_int</span> <span class="o">=</span> <span class="nb">pow</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">PIN_DIGITS</span><span class="p">)</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">pin</span> <span class="o">=</span> <span class="n">randbelow</span><span class="p">(</span><span class="n">max_int</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">db</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">Query</span><span class="p">()</span><span class="o">.</span><span class="n">id</span> <span class="o">==</span> <span class="n">pin</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">result</span><span class="p">:</span>
<span class="n">db</span><span class="o">.</span><span class="n">insert</span><span class="p">({</span><span class="s2">"id"</span><span class="p">:</span> <span class="n">pin</span><span class="p">,</span> <span class="s2">"conference"</span><span class="p">:</span> <span class="n">conference</span><span class="p">})</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span>
<span class="s2">"message"</span><span class="p">:</span> <span class="s2">"Successfully retrieved conference mapping"</span><span class="p">,</span>
<span class="s2">"id"</span><span class="p">:</span> <span class="n">pin</span><span class="p">,</span>
<span class="s2">"conference"</span><span class="p">:</span> <span class="n">conference</span><span class="p">})</span>
</code></pre></div>
<p>That will give us back what Jitsi Meet expects.</p>
<p>If you're wondering what limebrass is, it's the name I gave to my
conferencing system. It doesn't mean anything other than it's a unique
name. </p>
<p>Now we must tell Jitsi Meet to use this new mapping. That's done by
editing the <code>CONFIG/web/config.js</code> file and adding in
<code>dialInConfCodeUrl</code> in the large Javascript object, before the
<code>makeJsonParserHappy</code>, such as:</p>
<div class="highlight"><pre><span></span><code> dialInConfCodeUrl: 'https://example.com/conferenceMapper',
</code></pre></div>
<p>Now that this is done, we need to turn our attention back to Twilio
for a moment and how we will connect the PIN we've just made to the
phone system.</p>
<p>Luckily for us, Twilio makes this very easy with a <code>Gather</code> directive
that can be used to collect digits. Our process will be to ask the
caller to enter in their PIN, then if the conference exists, they'll
be connected into it. If not then they'll be given another chance to
enter their PIN. And if they can't do it three times, they'll be asked
to call back.</p>
<p>Twilio's <code>Gather</code> directive works a bit like an HTML form in that it
has an <code>action</code> paramater that it POSTs the result to.</p>
<p>If we didn't care about letting someone try to enter their pin a
second or third time, we could use one single endpoint for both the
answer and the gather, but since we do want to allow this, we'll need
two endpoints.</p>
<p>First let's change our <code>answer</code> code and add the redirect.</p>
<p>Our first step then will be to change our <code>/answer</code> code to announce
that the user is in the phone conference, then to redirect them to the
gather request.</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nv">@app</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="ss">"/answer"</span><span class="p">)</span>
<span class="w"> </span><span class="n">def</span><span class="w"> </span><span class="n">answer</span><span class="p">()</span><span class="err">:</span>
<span class="w"> </span><span class="ss">"""Announce the conferencing system"</span>
<span class="w"> </span><span class="n">resp</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">VoiceResponse</span><span class="p">()</span>
<span class="w"> </span><span class="n">resp</span><span class="p">.</span><span class="n">say</span><span class="p">(</span><span class="ss">"Welcome to the conferencing system!"</span><span class="p">)</span>
<span class="w"> </span><span class="n">resp</span><span class="p">.</span><span class="n">redirect</span><span class="p">(</span><span class="ss">"/gather?tries=0"</span><span class="p">)</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nf">str</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
</code></pre></div>
<p>You may have noticed that I added a query parameter <code>tries</code> to the
URL. That's so we can count the number of tries that have been
attempted and hang up when it's been too many.</p>
<p>Now let's work on the <code>gather</code> code.</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nv">@app</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="ss">"/gather"</span><span class="p">)</span>
<span class="w"> </span><span class="n">def</span><span class="w"> </span><span class="n">gather</span><span class="p">(</span><span class="n">methods</span><span class="o">=[</span><span class="n">"GET", "POST"</span><span class="o">]</span><span class="p">)</span><span class="err">:</span>
<span class="w"> </span><span class="ss">"Gather the PIN number"</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">request</span><span class="p">.</span><span class="k">method</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="ss">"GET"</span><span class="err">:</span>
<span class="w"> </span><span class="n">tries</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nc">int</span><span class="p">(</span><span class="n">request</span><span class="p">.</span><span class="n">args</span><span class="p">.</span><span class="k">get</span><span class="p">(</span><span class="ss">"tries"</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">))</span>
<span class="w"> </span><span class="n">resp</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">VoiceResponse</span><span class="p">()</span>
<span class="w"> </span><span class="n">gather</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Gather</span><span class="p">(</span><span class="n">num_digits</span><span class="o">=</span><span class="n">PIN_DIGITS</span><span class="p">,</span><span class="w"> </span><span class="k">action</span><span class="o">=</span><span class="ss">"/gather?tries={tries})</span>
<span class="ss"> gather.say("</span><span class="n">Please</span><span class="w"> </span><span class="n">enter</span><span class="w"> </span><span class="n">your</span><span class="w"> </span><span class="n">conference</span><span class="w"> </span><span class="n">number</span><span class="p">,</span><span class="w"> </span><span class="n">followed</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">pound</span><span class="w"> </span><span class="nf">sign</span><span class="p">.</span><span class="ss">")</span>
<span class="ss"> resp.append(gather)</span>
<span class="ss"> # If no response, end the call</span>
<span class="ss"> resp.say("</span><span class="n">I</span><span class="w"> </span><span class="n">didn</span><span class="s1">'t a conference pin. Please call back once you have it!")</span>
<span class="s1"> return str(resp)</span>
<span class="s1"> else:</span>
<span class="s1"> # This is the POST method, and should only be called once a</span>
<span class="s1"> # gather is made</span>
<span class="s1"> tries = int(request.args.get("tries", 1))</span>
<span class="s1"> pin = int(request.form.get("Digits", 0)</span>
<span class="s1"> if not pin:</span>
<span class="s1"> resp.say("I didn'</span><span class="n">t</span><span class="w"> </span><span class="k">get</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="n">conference</span><span class="w"> </span><span class="n">pin</span><span class="p">.</span><span class="w"> </span><span class="n">Please</span><span class="w"> </span><span class="k">call</span><span class="w"> </span><span class="n">back</span><span class="w"> </span><span class="n">once</span><span class="w"> </span><span class="n">you</span><span class="w"> </span><span class="n">have</span><span class="w"> </span><span class="n">it</span><span class="err">!</span><span class="ss">")</span>
<span class="ss"> return str(resp)</span>
<span class="ss"> # Look up the PIN</span>
<span class="ss"> result = db.search(Query().id = pin)</span>
<span class="ss"> if not result:</span>
<span class="ss"> tries += 1</span>
<span class="ss"> if tries >= 2:</span>
<span class="ss"> resp.say("</span><span class="n">Too</span><span class="w"> </span><span class="n">many</span><span class="w"> </span><span class="n">incorrect</span><span class="w"> </span><span class="n">pin</span><span class="w"> </span><span class="n">attempts</span><span class="p">.</span><span class="w"> </span><span class="n">Please</span><span class="w"> </span><span class="k">call</span><span class="w"> </span><span class="n">back</span><span class="w"> </span><span class="n">once</span><span class="w"> </span><span class="n">you</span><span class="w"> </span><span class="n">have</span><span class="w"> </span><span class="n">it</span><span class="err">!</span><span class="ss">")</span>
<span class="ss"> resp.rediect(f"</span><span class="o">/</span><span class="n">answer</span><span class="vm">?</span><span class="n">tries</span><span class="o">=</span><span class="err">{</span><span class="n">tries</span><span class="err">}</span><span class="ss">", method="</span><span class="k">GET</span><span class="err">'</span><span class="p">)</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nf">str</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
<span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="n">Success</span><span class="err">!</span><span class="w"> </span><span class="n">Redirect</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">caller</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">correct</span><span class="w"> </span><span class="n">conference</span><span class="err">!</span>
<span class="w"> </span><span class="n">conference</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">result</span><span class="o">[</span><span class="n">0</span><span class="o">][</span><span class="n">"conference"</span><span class="o">]</span>
<span class="w"> </span><span class="n">dial</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Dial</span><span class="p">()</span>
<span class="w"> </span><span class="n">dial</span><span class="p">.</span><span class="n">sip</span><span class="p">(</span><span class="n">f</span><span class="ss">"sip:{SIP_USERDOMAIN}?X-Room-Name={conference}"</span><span class="p">)</span>
<span class="w"> </span><span class="n">resp</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">dial</span><span class="p">)</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nf">str</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
</code></pre></div>
<p>Phew! Our little Python program is getting bigger, but it's all
relatively straightforward code.</p>
<p>You may notice that I'm playing a little fast and loose with error
handling here. That's because this application will only be interacted
with by other known applications. If an exception occurs, it's due to
a bug somewhere, rather than us wanting to try to correct for it. This
is also why I don't feel very strongly about disabling the Debug mode,
though if I ran this for any significant installations, I would turn
it off.</p>
<p>At this point, a user who knows a conference pin can dial in. But how
will they know the number to dial into? That's the next section!</p>
<h1>Setting Call-In Number</h1>
<p>Now that we have the pin sorted out, let's make it easy for someone to
find the call-in number(s). Jitsi Meet makes it easy to find out by
having a configurable url that returns a JSON document with a list of
phone numbers. This could be a static file, but let's just include it
in our web application.</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nv">@app</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="ss">"/dialInNumbers"</span><span class="p">)</span>
<span class="w"> </span><span class="n">def</span><span class="w"> </span><span class="n">dial_in_numbers</span><span class="p">()</span><span class="err">:</span>
<span class="w"> </span><span class="ss">"""Return our available phone numbers"""</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">jsonify</span><span class="p">(</span><span class="err">{</span>
<span class="w"> </span><span class="ss">"message"</span><span class="err">:</span><span class="w"> </span><span class="ss">"Phone numbers available."</span><span class="p">,</span>
<span class="w"> </span><span class="ss">"numbers"</span><span class="err">:</span><span class="w"> </span><span class="n">PHONE_NUMBERS</span><span class="p">,</span>
<span class="w"> </span><span class="ss">"numbersEnabled"</span><span class="err">:</span><span class="w"> </span><span class="k">True</span><span class="err">}</span><span class="p">)</span>
</code></pre></div>
<p>In this code, we use our environment to set the phone numbers we want
to use. The format used is a JSON object. Showing an example is
probably easier than explaining it:</p>
<div class="highlight"><pre><span></span><code> {"US":
["+1.555.555.1212"]}
</code></pre></div>
<p>You can see we have a mapping of country codes and a list of
numbers. The formatting of the numbers is entirely up to you.</p>
<h1>Setting Call-Out</h1>
<p>At this point we can do everything a standard call-in phone conference
can do, but we can also optionally allow for call-outs, which is to
say that we can initiate a phone call from inside a conference.</p>
<p>This can be useful if you're needing to contact someone directly and
don't want to go through the dance of having them call in. But because
this can also be used to initiate calls, it's advised that this only
be enabled on Jitsi installatiosn that have authentication turned on!</p>
<p>With that warning out of the way, let's make a new endpoint!</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="nv">@app</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/callOut'</span><span class="p">)</span>
<span class="w"> </span><span class="n">def</span><span class="w"> </span><span class="n">call_out</span><span class="p">()</span><span class="err">:</span>
<span class="w"> </span><span class="ss">"Make an outgoing call"</span>
<span class="w"> </span><span class="n">caller_id</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">request</span><span class="p">.</span><span class="n">args</span><span class="o">[</span><span class="n">'callerId'</span><span class="o">]</span>
<span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">request</span><span class="p">.</span><span class="n">args</span><span class="o">[</span><span class="n">'To'</span><span class="o">]</span>
<span class="w"> </span><span class="n">to_formatted</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">to</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)</span><span class="o">[</span><span class="n">0</span><span class="o">]</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s1">':'</span><span class="p">)</span><span class="o">[</span><span class="n">1</span><span class="o">]</span>
<span class="w"> </span><span class="n">resp</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">VoiceResponse</span><span class="p">()</span>
<span class="w"> </span><span class="n">resp</span><span class="p">.</span><span class="n">dial</span><span class="p">(</span><span class="n">to_formatted</span><span class="p">,</span><span class="w"> </span><span class="n">caller_id</span><span class="o">=</span><span class="n">caller_id</span><span class="p">,</span><span class="w"> </span><span class="n">answerOnBridge</span><span class="o">=</span><span class="k">True</span><span class="p">)</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nf">str</span><span class="p">(</span><span class="n">resp</span><span class="p">)</span>
</code></pre></div>
<p>As you can see, it takes in two arguments, <code>To</code> and <code>callerID</code>. <code>To</code>
contains a full SIP address, so what we need to is strip that out so
it looks like a phone number, formatted in <a href="https://en.wikipedia.org/wiki/E.164#Numbering_formats">E.164</a> format, ie a <code>+</code>
symbol, then the country code and phone number.</p>
<p>Twilio's policy is that the <code>calledId</code> must be a number we have
associated with our account, ether that we bought from them or have
verified. We'll supply that manually, but we could also be clever here
and look at other factors in deciding which caller ID to supply. For
example, we might have numbers in different countries and want to use
the appropriate number for the country we're dialing out to. As long
as the number is either through Twilio or verified with them, we can
do that. In this case, though, I've simply supplied the calledId as an
argument to the script in Twilio's SIP domain configuration.</p>
<p>The final bit of setup, then, is to go to your SIP domain (ie
<a href="https://www.twilio.com/console/voice/sip/endpoints">https://www.twilio.com/console/voice/sip/endpoints</a>?), clicking on your
SIP domain, then putting in the URL (ie
<code>https://example.com/callOut?calledId=+15555551212</code>) in the "A CALL
COMES IN" field.</p>
<p>That may seem a little confusing at first, but we need to think about
it from the perspective of the SIP endpoint. It is what's getting a
call, which is why it's considered "inbound" for it.</p>
<h1>Final thoughts</h1>
<p>And that's it! A fully functional system for both calling in and
calling out with Twilio and Jitsi! All we had to do was write a tiny
amount of glue code and viola, we have a powerful connection between
our phone system and a conference system! If you don't use it much
(like me) then the price for this is going to be fairly inexpensive
and we didn't have to set up a PBX server like Asterisk, just a little
web server!</p>
<p>There's certainly a lot more to do here if you want to turn this
little toy into a "real application". You'll want to change the voices
in your call-in to something pleasant, you'll want to set up real
logging, and probably a more substantial database than what we have,
but this should be a good launching point for a beginner.</p>
<p>Enjoy!</p>Why OpenStreetMap is in Serious Trouble2018-02-16T00:00:00-05:002018-02-16T00:00:00-05:00Serge Wroclawskitag:blog.emacsen.net,2018-02-16:/blog/2018/02/16/osm-is-in-trouble/<p>I was a contributor for OpenStreetMap for a long time, and I advocated for OpenStreetMap for a long time, but the project has stalled while the proprietary mapping world has continued to improve in data quality. For those of us who care about Free and Open data, this is a …</p><p>I was a contributor for OpenStreetMap for a long time, and I advocated for OpenStreetMap for a long time, but the project has stalled while the proprietary mapping world has continued to improve in data quality. For those of us who care about Free and Open data, this is a problem. In this article, I explore the reasons why I think OSM has stalled, as well as solutions to get the project back on track.</p>
<p>I was a contributor to the OpenStreetMap project from 2008 until roughly 2016. I was heavily invested in the project. In that time, I mapped, organized mapping groups in two cities, contributed to the founding of OpenStreetMap US, a non-profit dedicated to OpenStreetMap in the United States, gave talks on and about OpenStreetMap, contributed to the OpenStreetMap.org codebase, mentored two students for OSM through Google Summer of Code, started a working group dedicated to import of data into OpenStreetMap in the US, created and coordinated a massive bot run in the US, moderated the Reddit page <a href="https://reddit.com/r/openstreetmap">r/OpenStreetMap</a> and was a member of the OpenStreetMap Data Working Group, which gave me escalated privileges (both politically and technically) for the project. There are few parts of OpenStreetMap where I didn't have some either direct or indirect involvement.</p>
<p>I'm also the author of an article called <a href="https://blog.emacsen.net/blog/2014/01/04/why-the-world-needs-openstreetmap/">Why the World Needs OpenStreetMap</a>, which appeared in two large publications, The Guardian Online and Gizmodo, was on the front page of Hacker News twice and translated into at least four different languages. I was a proud OpenStreetMap advocate!</p>
<p>Before I criticize the project, I want to state emphatically that I still believe wholeheartedly in the core principles of OpenStreetMap. We need a Free as in Freedom geographic dataset just as much today as we did in the past. When I wrote my article about OSM in 2012, self-driving cars and other services were still a dream. Today the importance of having a highly accurate, libre geographic dataset is more important than ever, and I support those working to make it happen.</p>
<p>That said, while I still believe in the goals of OpenStreetMap, I feel the OpenStreetMap project is currently unable to fulfill that mission due to poor technical decisions, poor political decisions, and a general malaise in the project. I'm going to outline in this article what I think OpenStreetMap has gotten wrong. It's entirely possible that OSM will reform and address the impediments to its success- and I hope it does. We need a Free as in Freedom geographic dataset.</p>
<p>As long as this post is, it's not a comprehensive list of all the problems with the project, only the ones I found most directly affect the project's success and that I wasn't able to address myself during my time on the project.</p>
<h2>When the World Needs a Map, Give them a Database</h2>
<p>The first problem that I feel plagues OSM is that the OpenStreetMap Foundation views the mission of the project to provide the world a geographic database, but not geographic services. OSM gives people the tools to create their own map rather than offering them a simple, out of the box solution. Providing the ability for individuals and organizations to make their own map may work well for some, but it discourages small and medium size organizations from using OSM and thus engaging with the project. And even if they do use our data, their engagement is through a third party, rather than directly with us.</p>
<p>When you go to OpenStreetMap.org, you see a map and a few extras, such as a search window, along with a few extra buttons such as "Log In" and "Edit." It would be reasonable to assume that OpenStreetMap is a map, like Google Maps or other map projects, but while there is a map on OpenStreetMap.org, OpenStreetMap doesn't want you to use it. Instead, they want you to use the information from OpenStreetMap to make your own map, or find someone else to make the map for you.</p>
<p>If you find this strange or confusing, you're not alone.</p>
<p>A map is nothing more than a visualization of a collection of facts. We can understand this in terms of geometry. Let's imagine a furniture store called Frita's Furniture. Our map is a simple two-dimensional plane just like we had in geography class, and it's at 10,10. We might also imagine a road called Main Street that runs from location 2, 9 all the way down to 15, 9.</p>
<p>The location of the store and the road are geographic facts, but if we wanted to represent this data visually, we'd typically use a map. We'd choose just how to draw the road. Would we use a simple line or a more road-line picture? How wide should the line be? What color would it be? Where would we put the name of the road? We could put it across the line, or alongside it, or some other way entirely. And do we want to represent the store as a dot, or an icon of a store?</p>
<p>Years ago, map makers would handle this process manually, but with computers, we generally call this process map rendering, and there are many decisions around the rendering of a map, such the usage of the map, local conventions and even just keeping within a given organization's map style.</p>
<p>But most people don't care about any of this. They just want a map. OSM has a map on its website but discourages its use by third parties. Instead, users are expected to either find a commercial service to render the map for them or else do it themselves.</p>
<p>The project leaders claim that this is because they want people using OpenStreetMap to understand the difference between the geographic data and its visual representation and to encourage a free market ecosystem of rendered map providers, but it's also a fact that many of the individuals who push for this separation also sell commercial map services. I explore this conflict of interest later in this post.</p>
<h3>Unclear Usage Policies</h3>
<p>I mentioned earlier that OSM discouraged use of its maps on other websites. It does this through technically enforced usage policies. Understanding a usage policy is usually a straightforward process. An individual or organization gets permission to use a service a certain amount. We could imagine this being done by the number of map requests, or by bandwidth, etc. But OSM's usage policy is entirely different. They allow but discourage the use of the free map and then disallow any single application that is using over 5% of the map bandwidth. This policy is bizarre on several levels.</p>
<p>To understand why this is so strange, we can use an analogy. Let's imagine that I make ice cream. I put the recipe for my ice cream outside my house and suggest people make their own. I also offer free samples out of my home. Above my door, I hang a sign saying "Please don't ask for free samples". Then when people come in and ask for a sample, I give it to them. People may spread the word about my free ice cream and suggest their friends use it. Imagine we have a person named Fred who is a fan of my ice cream and recommends that all his friends go to my house for ice cream. I continue to dish out free ice cream to anyone who asks. But if one individual like Fred refers too many people to me, I will cut off access to everyone Fred sent.</p>
<p>Furthering this analogy, I will tell Fred's friends that they've eaten too much free ice cream, instead of telling Fred. And how many people is too many people? The answer for OpenStreetMap is anything over five percent of the total amount of free ice cream that I've dished out that day. Fred has no idea how many people I've served, so the only thing he can do is ultimately not refer people to my house.</p>
<p>This analogy works because no since no single service knows what any other service is doing, there's no way to know how many other applications have requested how many map requests. Also, since the top services and applications will change over time, you may be fine one day and in trouble the next. Again, there's no way to know. And when you do cross the line of using the service too much, your users would get an unfriendly message about unavailability, not you.</p>
<p>OSM could create standard usage policy, spelling out exactly how much free usage is allowed. It could also choose to create "premium membership" and encourage people to use its tile service (rendered map) service, but right now, using OSM tiles without going through a third party is hard.</p>
<h3>A Bad Geocoder</h3>
<p>When you type an address into a map and it gives you the location, that is called Geocoding. When your GPS or phone knows where you are and gives you a building or street address, that is called Reverse Geocoding. The geocoder featured on OpenStreetMap.org is called Nominatim, and it's awful.</p>
<p>Nominatim is not the only OSM geocoder. Much like the map rendering, it is possible to write your own or use a commercial geocoding service. But Nomatim is the most popular geocoder available for OpenStreetMap, it's used on the website and Nominatim is the service that is listed on OpenStreetMap.org under its APIs.</p>
<p>In its defense, let me say that Geocoding is hard and Nominatim itself is quite complex; its almost a feat of engineering. The developers who work on it put enormous effort into writing Nominatim. The problem is that such software needs to be maintained or sometimes replaced entirely to be useful. While there have been Nominatim maintainers, it's not been given the time or attention that it needs and deserves.</p>
<p>To understand why Nominatim is bad, one has to understand how most people use a geocoder. They're most often looking up a business or something vague like "Staples downtown Springfield". That simple three letter query is asking quite a bit. It is asking the computer to know what Springfield is and to limit the query to that. It's then asking to limit it to (or near!) an area called "downtown", and finally, it's limiting results to Staples.</p>
<p>But Nomintim can't handle such queries. It can barely handle simple address queries, such as "123 Main Street". As an example, if I typed an address into Nominatim near my location in New York City, it might come up with a result in Iowa, which for whatever reason, it's more inclined to offer me.</p>
<p>If I try to specify my location as "Manhattan", as of the time of this writing Nomintim will first assume that I mean Manhattan, Kansas, ignoring both the prominence of Manhattan in New York and the fact that the query itself is originating from New York City. Worse still, it's not possible to search for intersections. If I type "53rd and 6th, New York City" into Nominatim, it doesn't understand it. Even if I try to refine the search as "53rd Street and 6th Avenue, New York City", it doesn't work. Intersections are not addresses to Nominatim. It doesn't understand stores, or "near" or categories such as "restaurant". The results it comes up with are often irrelevant, and the service is quite slow.</p>
<p>While other geocoders for OSM exist, such as Pelias and Photon, only Nominatim is run and supported by the OSM Foundation.</p>
<h2>No moderation/review model</h2>
<p>One of the most significant technical problems with OSM is the lack of a review model, that is for a change to the map to be staged and then reviewed before being applied. Not having this functionality caused ripples of problems throughout the system, some of which I'll discuss here.</p>
<h3>New mapper problem</h3>
<p>Editing on OSM can be challenging for a beginner, and as the project tried to attract new mappers (editor contributors), we would run into people who just mapped incorrectly. Unfortunately, because OSM's data model doesn't include a review stage bad edits are committed to the map and often left undiscovered, or even if they're removed, the original editor doesn't usually see why.</p>
<p>Having the ability for a mapper to contribute changes and then have those changes be reviewed would have potentially left the map with higher quality data and a sort of mentorship model between new contributors and more experienced editors.</p>
<p>I hoped that this would be improved when I mentored a feature placed into OpenStreetMap called "Changeset Comments", in which users could leave feedback for one another's changes. Unfortunately, this ended up not being something many people used constructively, and it was a mess.</p>
<h3>Without Moderation, Bots are Hard</h3>
<p>Bots could be very useful in OSM in finding mistakes caused either by inaccurate data sources or by editing blunders. For example, if there was a road named "Main Street" connected to another road called "Main Stret," it was likely a spelling error and should be corrected. But it would be a good thing if changes were reviewed by a human being first.</p>
<p>But since OSM doesn't provide any mechanism for reviewed edits, these kinds of suggested changes don't exist. Either bot edits are executed without oversight, which could lead to errors, or they aren't done at all and the project misses out.</p>
<h3>Imports are Difficult</h3>
<p>Imports are challenging for a variety of reasons, but having a moderation or review model would make things much easier by allowing changes to be staged. The inability to stage and review massive changes to the map have caused problems in the past and many bad imports go unnoticed. If the project instead required that a human review edits before being committed to the system, bad imports could be detected before they cause problems.</p>
<p>Due to the lack of staging inside OSM itself, staging systems have been written for other OSM related projects. These systems often require editors to then manually place those changes in OSM. This process is labor intensive and makes some imports so challenging that they die before they begin.</p>
<h3>Vandalism is hard to manage</h3>
<p>Like Wikipedia, OSM has people who purposefully vandalize the project. Vandals have a variety of motivations. Some vandals are your run of the mill Internet trolls who enjoy causing problems. Sometimes a company wants something on the map despite community consensus against it and change the map to suit their needs, even if the project as a whole is against it. Sometimes mappers use OSM to make political statements such as in the case of disputed territories, and sometimes a geospatially based game, such as Pokemon Go, will use OSM to generate its data and players find that they can change the map to gain an advantage. Whatever the reason, OSM has vandals.</p>
<p>Vandalism is difficult in OSM because without a moderation system, it has to be "cleaned up" rather than prevented in the first place. Detecting vandalism is difficult. Several people ran monitoring tools to try to find problematic edits and imports. I was one of those people.</p>
<p>Even if problematic edits are detected, removing then means making even more edits. The history of the project is littered with lots of small changes that are only there to remove some previous change. Worse still, if the vandalism isn't detected early then someone else might modify an object that was previously vandalized, creating a situation in which either a tool or a person would have to separate the good edits from the bad, a manual process that can be labor intensive.</p>
<p>A moderation tool would prevent much of this. Many vandals would find that their work would not get into the database and would move on. While some malicious edits would still get through, we would be able to address a majority before it became a problem.</p>
<h3>External Tools are Hard</h3>
<p>One of my contributions to OpenStreetMap was working to improve MapRoutlette, a tool which helps find problems in OSM and offers users the opportunity to fix them. One feature that we wanted in MapRoutlette was to be able to present users with simple "Yes/No" type questions. Unfortunately, while not impossible this would have been a complicated task in OpenStreetMap. If these edits could have gone to a moderation queue, we could have been more confident, and possibly not needed MapRoutlette at all in some cases.</p>
<p>Many developers wanted to solve this same problem, offering the ability to add helpful but anonymous edits to the project. But since OpenStreetMap requires every edit be committed by an individual user, rather than a company or bot account, the barrier of entry for casual mappers was often too high.</p>
<h2>OSM's Lack of Layers</h2>
<p>Most geographic databases use a layered approach to represent different features. One layer may represent political boundaries; another may represent the road network, a third may represent water features, and so on.</p>
<p>Instead of the traditional layers, OSM chooses to use a single layer and then tags (key/value pairs) on individual objects. At first this seems like a good idea, but ultimately it ends up creating a huge mess.</p>
<h3>Tools are Harder to Write</h3>
<p>Imagine if we wanted to write an editor for OSM that only worked with the road network. This task would seem straightforward in that we would just need to extract features that correspond are tagged as road, such as <code>highway=*</code>. Unfortunately, it's not that simple.</p>
<p>First, an editor that edits these road features must not only pick up the roads (<em>ways</em> in OSM terminology) but also the points (<em>nodes</em>) that make up that road. Secondly, if the road is particularly complicated, it may be represented as a <em>relation</em>. Editing this way is time consuming but straightforward.</p>
<p>What is not as straightforward is that a feature such as a road may also be playing double-duty as another feature, such as a political boundary, as may any of its associated features. Editing roads may inadvertently result in changing a political boundary.</p>
<p>Having map features that represent such radically different meaning puts an onus on both tool makers and the individual editor working on OSM to be aware of any changes they make possibly having consequences that go beyond what they think they're doing.</p>
<h3>Imports Are Difficult Without Layers</h3>
<p>One of the keys of the Free and Open Source software movements have been code reuse, the idea that you can integrate software together from different sources and have it work seamlessly together. One would think that it would be much the same with geographic data, but because of the lack of layers, it's very challenging to import data into OSM.</p>
<p>Without layers, it's difficult to extract a specific region by feature and analyze or replace that. Instead, because of its complex tagging system, it needs to be analyzed as a whole. Imports are possible but made more difficult without layers to make the job of data analysis by isolation easier.</p>
<h2>No Support For Observational, or Other Datasets</h2>
<p>One of the core tenants of OpenStreetMap is that it only stores persistent, personally verifiable data. The only exceptions to this are political boundaries- and even these exceptions can be problematic. Unfortunately, this also presents a problem with third parties want to use OSM for things outside of the project scope.</p>
<p>As an example, let's take Pokemon Go. Pokemon Go is an augmented reality game in which real-life features are connected with imaginary creatures in which the player must battle and collect. The frequency and location of where these creatures appear is based on various map features.</p>
<p>Pokemon Go players wanted to use OSM to document the location of creatures to make it easier for other players to find rare creatures and improve their collection. OSM disallows this kind of data in the same way that it might for bird watchers- while it's interesting, the impermanence of this data made it a poor candidate for the project and thus the data would be immediately removed.</p>
<p>But it doesn't have to be something as trivial as a game- layers could also allow other specialized data such as potholes, red light cameras or even bird or animal sightings. It would make the project useful to many more people.</p>
<h2>Lack of Permanent IDs</h2>
<p>In any database, objects have an ID field, usually a numeric value to look the record up by. OSM is no different and every object inside OSM has an ID field. Unfortunately, in OSM the ID fields represent the low-level objects rather than any high-level concept. This creates a huge problem. I will call this idea a "Conceptual Object", and show how the lack of permanent IDs for them is problematic.</p>
<h3>Diving Deeper</h3>
<p>To understand why the lack of permanent IDs is a problem, we have to dive a little deeper into how OSM works. While many of these low-level details are beyond the scope of this article, I will present the basics of how OSM stores information. A point in OSM is called a <em>node</em>, and every node has an ID. Points may be collected into a line, and that line is called a <em>way</em>, and collections of nodes and ways may be combined into a more complex object called a <em>relation</em>. A relation may also contain other relations. Nodes, ways, and relations all have ID fields.</p>
<p>To illustrate this, let's think of a building, that building has properties. The building is at a certain location, it's a specific size, shape, and has an address. If it's large enough, it may have multiple addresses. But the concept of the building is unified. In OSM, that building could be represented by a single node, representing the address. Or a building may be represented by a way of the building outline, or a building may be represented by a relation, encompassing details of the various building elevations, levels, and roof types. The problem is that if I'm doing a lookup, there is no straightforward way to ask about the building. Instead, I will have to look at aspects of the building, such as its address, or its location.</p>
<h3>History Lost</h3>
<p>As strange as this may seem, it's entirely possible in OpenStreetMap to take a node from one side of the world, move it to the other side of the world and use it for something else entirely. For example, it is technically possible to take a part of a house, move it to another continent and use it as part of a road. While this is highly unusual, this is not disallowed. If I look at the history of the node, I will see it move. While this may keep the history of the element, it does not keep the conceptual history of an object.</p>
<p>For example, if we start with a representation of an object being the building as a single node, then move to a complex relation, that won't be reflected in the object history, and thus the changes over time are lost.</p>
<p>Permanent IDs on conceptual objects could help with this by providing a history of what the data represents rather than just the data itself.</p>
<h3>Import Conflation</h3>
<p>Amongst other problems, not having a permanent ID for a conceptual object in OSM is the challenging of conflating objects in OSM to objects in other datasets. For example, if we're given a building database from a local government, each building in that dataset will have an ID. We will want to compare that ID to our existing objects. Unfortunately, to do that, we're left with two choices- either we create a new identifier (key) in which to do the conflation, or we have to use the second dataset's ID inside OSM- neither of which is an optimal solution.</p>
<h3>It's Hard to Build Connections to Other Datasets</h3>
<p>Many people have envisioned projects that connect to OSM to offer reviews or other data associated with OSM, but without a permanent ID, this is not practical. Objects in OpenStreetMap may contain some data such as cuisine type or opening hours along with the name and address, but the review site will need to be able to have a permanent link to objects in OSM, which it can't currently.</p>
<h2>No Standards in Data Representation</h2>
<p>In OpenStreetMap, there are no formal standards in the project for the representation of features on the map. As an example, let's take the example of a sidewalk. Sidewalks are useful things to have on a map because they tell us if the road is pedestrian friendly. Sometimes sidewalks are represented by an attribute on the road itself. Sometimes sidewalks are represented as a line (way) that runs parallel to the road. Sometimes those ways have the name of the street as their own name, and sometimes they don't have any name at all.</p>
<p>If you are a mapper, this is confusing, since there's no one standard way to map things. If you're trying to build tools to work with OpenStreetMap, the lack of standardization of data across the project makes it challenging to work with as a whole.</p>
<p>There is an informal process for data representation, mainly done on the Wiki, but because this isn't formally enforced, and changing data en mass may be considered a form of vandalism, data consumers are forced to write tools that accept many representations of the same data.</p>
<h2>The APIs are Slow to Evolve</h2>
<p>As of writing, the current official OpenStreetMap API is 0.6. The API hasn't version hasn't changed since 2009. While a stable API can be a good thing for a mature software project, in the case of OpenStreetMap, this is as more a reflection of poor project management.</p>
<p>An API is part of a protocol that either allows a client to talk to a server or for servers to communicate with each other. In this case, we're talking about OpenStreetMap's editing API which is used between OSM and editing software.</p>
<p>The OpenStreetMap editing API is very powerful and complete, but it has some design choices that made sense in 2009 that have largely been replaced by better technical options in nine years since. These include small changes, such as the data serialization format, as well as more significant changes such as the internal data representation.</p>
<p>As an example, in 2012 there have been several proposals made to create a new datatype called an a <em>area</em> that would greatly simplify the representation of certain types of geographic features. Despite this and the offer of technical help, the project has not made any significant progress on this or other important technical issues.</p>
<h2>OSM has Hidden Gatekeepers</h2>
<p>Dovetailing on the previous section, we have to ask why the project has not made more technical progress, and the answer is that sadly the keys to the OSM castle largely do not lie in the hands of the OpenStreetMap Foundation, but instead in the hands of one or two individuals who act as gatekeepers to the project's source code and infrastructure.</p>
<p>While it's not uncommon for a Free Software or Open Source project to have a "Benevolent Dictator for Life", these roles are often replaced by a more formal structure as the needs of the project grow. In the case of OpenStreetMap, there is a formal entity which owns the data, called the OpenStreetMap Foundation. But at the same time, the ultimate choices for the website, the geographic database and the infrastructure are not under the direct control of the Foundation, but instead rest largely on one individual, who (while personally friendly) ranges from skeptical to openly hostile to change.</p>
<p>As a former professional system administrator, I relate strongly to these types of individuals. At the same time, the desires of them need to be balanced by the overall needs of the project to make progress and keep momentum to keep its userbase happy and engaged.</p>
<p>That is not the case here, and it's to the detriment of the project.</p>
<h2>The OpenStreetMap Foundation Culture</h2>
<p>It would be easy to think about the OpenStreetMap Foundation (the OSMF) as similar to the Wikipedia Foundation, but aside from the high-level view of being the holder of Free Data, the two projects are managed radically differently.</p>
<p>The Wikipedia Foundation is a multi-million dollar organization that not only manages Wikipedia but other projects as well, such as the lesser known Wikidata and Wikinews. These projects aid in the organization's broad mission to provide high-quality information to the world. To serve this mission, Wikipedia spends a great deal of money on its infrastructure as well as directing and funding development of new tools for the community to use.</p>
<p>OpenStreetMap, on the other hand, relies primarily on donated hosting services and runs on a shoestring budget. It has no paid employees and does not fund or direct the development of its software base.</p>
<p>This has lead to some organizations trying to take up the mantle and improve the situation, including an organization that I helped found called OpenStreetMap US, which is a US based non-profit organization focused on promoting OSM in the United States. Among our goals for the organization was to fill in the gaps of development and mapping resources by the OSMF, which we partially succeeded in doing, but because of the fragmentation of organizations, we were less successful than we hoped.</p>
<p>In addition to OpenStreetMap US and other "chapters" around the world, there is the Humanitarian OpenStreetMap Team, whose mission it is to help promote OSM in developing nations and rally the OSM community during humanitarian crises. There is no reason that HOT needed to be an independent organization other than the unwillingness by the OSMF to expand its role. Even Steve Coast, one of the founders of OpenStreetMap saw and tried to address this problem with his organization, "Map Club."</p>
<p>The obvious question is why the OpenStreetMap leadership takes the positions that it does, despite the clear need for change. The answers in my view are commercialism in the project, along with a cultural desire to retain the feel of the project's early days.</p>
<p>While there are companies built around Wikipedia's engine (the Wikimedia Server), there are not many companies making money from repackaging Wikipedia. OpenStreetMap, on the other hand, has a commercial ecosystem around it, largely from the business of creating customized maps for customers.</p>
<p>Many of the founders of the project, as well as others, have launched commercial services around OSM. Unfortunately, this creates an incentive to keep the project small and limited in scope to map up the gap with commercial services which they can sell. This also applies to HOT, which has a financial incentive to get grant money for itself and not have those resources going to the OSMF.</p>
<p>In addition to these conflicts of interest is a desire to keep the project small in scope by senior members of the community who see the project as being about people and the mapping hobby and want to avoid imports or other activity that could be seen as removing the human factor from the project. They also see the dangers inherent in creating an organizational structure that demands money and fears it would create a perpetual cycle of needing to find donors simply to support a management layer.</p>
<p>I disagree and view the lack of a more active structure by the OSMF as the cause of the project's both stagnation and significant commercial influence.</p>
<h2>The World Had Changed</h2>
<p>When OSM was launched, governments did not release their data under free licenses. They only began doing so <em>because</em> OSM exists now as competition. Yet due to the problems I've outlined, OSM imports are difficult, and updating imports, once they get in OSM is nearly impossible. This is a critical problem for the project.</p>
<p>Similarly, when OSM was launched, drones were not cheap and available. AI wasn't able to do good visual detection of roads, and flying cars were still science fiction. Now all of these tools exist, and yet OSM is still stuck largely editing by hand.</p>
<p>If OSM relies exclusively on manual labor and be unable to work with other datasets, its data quality will continue to decline and the project will ultimately stagnate and fail.</p>
<h2>Just the Roadblocks</h2>
<p>It may appear at first that this article is a comprehensive list of everything I find wrong with OSM. It's not. There are many more concerns I have about the project, but I've limited my article to the scope of concerns I have that I feel are stopping the entire project from progressing. There will be time to fix the small issues if (and only if) the project as a whole succeeds. If it doesn't, then the small nit-picky problems are going to be irrelevant anyway.</p>
<p>It's my sincere hope that this article will be a call-to-action for OSM. There are many brilliant and inspiring individuals in the project. If I'm am a pun, I hope OSM will once again find its <em>way</em>.</p>Free Software Call To Action2016-11-27T11:28:00-05:002016-11-27T11:28:00-05:00Serge Wroclawskitag:blog.emacsen.net,2016-11-27:/blog/2016/11/27/free-software-call-to-action/<p>This last year has been a very difficult one for civil liberties and personal freedoms.</p>
<p>Donald Trump, the president-elect of the United States, has promised to round up and deport illegal immigrants as well as removing legal immigrants from the US. In addition, and more importantly, he's discussed creating a …</p><p>This last year has been a very difficult one for civil liberties and personal freedoms.</p>
<p>Donald Trump, the president-elect of the United States, has promised to round up and deport illegal immigrants as well as removing legal immigrants from the US. In addition, and more importantly, he's discussed creating a "Muslim Registry" and his cabinet has likened to the US Japanese Internment Camps of WW2, and supports mass surveillance. Pence, the vice-presidential nominee, supports rolling back protections for gays, lesbians and transgender people.</p>
<p>With these views espoused by the political leadership, it should be no surprise that hate crimes are on the rise, with the Southern Povery Law Center documenting over 700 hate crimes in just the week after the election.</p>
<p>Brexit symbolized a change for the United Kingdom, signaling a change from inclusion and connection with Europe to isolationism. Racist and xenophobic statements were part of the campaign to disconnect the UK from the European Union, drawing on ideas that in the past, the United Kingdom was strong, and could be again. In the last month, the UK has passed a law that gives the government not only the ability to track its citizens, but access to all their past activities online, including browser history, emails and other communications.</p>
<p>Aside from the US, right wing parties are on the rise in Europe, with the rising popularity of Frauke Petry's AfD party in Germany, Norbert Hofer's Austria Freedom party in Austria as well as Marie Le Pen's National Front in France. All of these political movements signal a move away from inclusion and social acceptance and towards isolationism, "traditional values" and the persecution of vulnerable populations such as refugees, immigrants, gays and lesbians, transgender people, people of color, Jews, Muslims, the physically handicapped or the neuro-divergent. If you aren't personally mentioned in this group, it's likely that you know someone who is- and believe me, they, I, am scared.</p>
<p>We're scared that our lives will be destroyed, that the work we've done to make our lives better will be undone, that marriages will be dissolved, that employment protections will be rolled back and we'll be fired, or worse, that we'll be forcibly taken from our homes, physically attacked or killed.</p>
<p>With these established facts, what practical steps do we take from here? I outline four practical steps that we Free Software people need to take.</p>
<ol>
<li>Establish and promote alternative communication platforms</li>
</ol>
<p>The first step the Free Software community needs to take is to recognize for ourselves that we need free methods of communication. Freedom is not free unless we can communicate with one another, whether that be messaging, audio/visual communication, or the distribution of static content.</p>
<p>Because of the enormity of this task, I've broken it down further into smaller sub-tasks.</p>
<p>A. Promote "small step" improvements in communication security.</p>
<p>We in the Free Software community have a tendency towards absolutism. There are understandable and justifiable reasons for that, but when the stakes are so high, we must not become so entrenched in [TK!] that we lose the opportunity to help others through smaller steps that they could take right now, without making a major change in their lives.</p>
<p>Asking all your friends to use OpenPGP and plain text communication might be asking too much, but sending people to <a href="https://whispersystems.org">Signal</a> and encouraging them to use it is a good first step. Similarly, encouraging the use of technologies like TOR, and pointing them to the <a href="https://www.eff.org/node/82654">EFF's Secure Messaging Scorecard</a> are great ways to make small improvements, which is our first step in making the kinds of progress that we need in ensuring our rights to communicate and congregate freely online.</p>
<p>B. Test, use and promote more comprehensive communication improvements</p>
<p>When a new communication platform launches and it respects your privacy, use it, and encourage others to use it. It may be difficult or impossible to get people to add something onto their existing communication platform, but with new platforms come "new rules" and it may be easier to encourage them to switch to a new system than retrofit their old one. You can help test these platforms and encourage their use by example.</p>
<p>C. Use and Promote Self-Hosted Communication Platforms</p>
<p>A big problem right now is the use of third party hosted platforms for our communication. Google and Facebook and others are very convenient, but result in a loss of privacy and control. Instead, self-hosted platforms (even on a commercial platform such as a VPS) are better as they decentralize your data.</p>
<p>If you are in a decision making capacity at your job, neighborhood association or other organization, encourage and support the use of self-hosted tools.</p>
<p>If you're a developer, consider the ease of use of deploying and maintaining the service for very small groups of 2-20 people. Use defaults that work well for small groups and consider small libraries such as SQLite instead of requiring a standalone database. If you can, provide a containerized version of your application for easy installation.</p>
<p>If you're a system administrator, consider helping provide helpful tutorials and scripts to make the deployment of software easier. </p>
<p>D. Offer support to alternative platforms</p>
<p>In the 90s, the community had social gatherings called Installfests where people came together to offer technical support and assistence in helping bring new people to Free Software. If we have entirely new operating systems or hardware, we may need to bring these Installfests back, not just for their role in technical assistance but also as an opportunity to make one on one, personal contact and build a local community. It's important for people to feel connected and part of something larger than themselves, and offering physical meetings can play an important role in realizing that goal.</p>
<ol>
<li>Recognize and support community diversity</li>
</ol>
<p>More than product developers, the Free Software community is built on top of a shared experience and shared culture. Part of that culture is acceptance and welcoming of people who are different from you. We need to stand by our community members, no matter where they're from, their religious background, their gender identity or sexual orientation. We are stronger together.</p>
<p>While these are nice sentiments, they must also be backed up with action, especially in regards to our most vulnerable populations. In some cases, that may mean codifying these principles into a community statement or code of conduct, but in all cases, it means that if you see something wrong, you need to stand up to it. We need to ensure that the feeling of camaraderie, familiarity and safety that most of us feel within the community is something shared with everyone.</p>
<ol>
<li>Communicate openly and loudly about the moral, legal and ethical threats to our liberties</li>
</ol>
<p>Free Software has always been about more than code, it's been part of a larger moral and ethical framework for understanding the world. We understand the role of Free Software in a larger [TK!] of personal liberties and freedoms. Free Software is a requirement for free communication in the online world. We need to know that our devices are doing what we want them to (and not what someone else might want them to do do). Once dismissed as paranoia, we know for a fact that modern "cloud enabled" devices transmit personal details about us, including our location, our interests and even record us.</p>
<p>If we value our freedom, we need to be able to establish boundaries around the encroachment on our civil liberties. In this time, when encroachment of civil liberties has become a norm, it is even more important to demonstrate the importance of this issue to other groups, and to build these principles into their mission statements and guidelines. To be clear, this is not about "Run Linux everywhere!", this is about codifying the principles of freedom and liberty in the framework of other organizations and by demonstrating the important of Free Software in carrying out that mission.</p>
<ol>
<li>Get involved in politics</li>
</ol>
<p>National politics is difficult and often inaccessible, but local politics is often easy to get involved in. Even small towns have many elected positions, and small changes even on a town or city level can make a difference, if only by bringing awareness to an issue. We must get involved in out political process, not by trying to change the minds of politicians from the outside but by being involved in the political process from the inside and using our influence.</p>
<p>This can be at every level. There are many articles online about getting involved in local politics, so I won't replicate their efforts here.</p>
<p>We must take a comprehensive approach to taking back our society. We owe it to ourselves and others to get involved not only to make our voices heard, but to protect ourselves and our loved ones.</p>
<p>We, the Free Software community, are a community of shared values and goals and now, more than ever, we need to turn those ideals into action.</p>Why is SFLC siding with Oracle over Linux developers?2016-02-28T20:36:38-05:002016-02-28T20:36:38-05:00Serge Wroclawskitag:blog.emacsen.net,2016-02-28:/blog/2016/02/28/why-is-sflc-siding-with-oracle-over-linux-developers/<p>I don't write about politics much on this blog (in fact I haven't
written anything in quite a while), but Free Software is something I
care a great deal about, and so when I see something fishy and
scandalous, I take interest, and that's exactly what's going on with
the …</p><p>I don't write about politics much on this blog (in fact I haven't
written anything in quite a while), but Free Software is something I
care a great deal about, and so when I see something fishy and
scandalous, I take interest, and that's exactly what's going on with
the Software Freedom Law Center's position on ZFS's license
compatibility with Linux.</p>
<!--more-->
<p>If you haven't heard, the Software Freedom Conservancy made a public
statement about ZFS. The facts can get a little complicated but in a
nutshell, Cannonical, the company that makes Ubuntu, is going to
distribute ZFS with the next release of their operating system
(16.04). Unfortunately, ZFS (which is owned by Oracle) is distributed
as a separate module under a licensed called the CDDL, which is
non-GPL compatible. You can read Conservancy's post here:
https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/</p>
<p>There's a simple solution to this problem. Since Cannonical can't ship
this code legally without infringing on the Linux (and possibly ZFS)
license, Oracle should simply change the licensing of ZFS from CDDL to
another, GPL-compatible license, either the GPL2 or a permissive
license such as the MIT license. It would be a relatively correction
for Oracle to make and it would allow the distribution of ZFS.</p>
<p>What's more interesting than this conflict is the response that the
Software Freedom Law Center posted an article on thier website about
the case:
https://softwarefreedom.org/resources/2016/linux-kernel-cddl.html</p>
<p>I imagine that if you've read it, you're probably thinking that's it's
incredibly well written, but you may feel vaguely dizzy. I understand;
I feel that way too when I read Eben Mogen's writing. His approach to
language can be incredibly inspiring, but at the same time by using so
many words, he creates a cloud of obfuscation of the facts. Usually
I'd say that I'm glad he's on our side, but this post of his is
decidedly not on our side.</p>
<p>For all its length, the article has two main points. Firstly, it says
that the CDDL (the licensed used by Oracle) is not compatible with the
GPL in fact, but it is so "in spirit". Secondly, it says that the
kernel developers are the problem, not Oracle.</p>
<p>To the first point, that the CDDL is close in spirit but not in fact
is an odd one for a lawyer to make. As former councel for the FSF,
Moglen has an understanding of the terms of the license that most
people don't have, and he knows very well that once the license is
committed to writing, we use what's written to decide whether or not
something is allowed under the license, rather than the ambigious idea
of "the spirit of the license", which he refers to in his post.</p>
<p>Which gets to Eben's second point, which is to shift the blame of the
problem onto the kernel developers, rather than onto Oracle. What he's
saying, in essence, is that he would like the GPL to be different than
it is. This reminds me of those people who argue that the GPL is a
de-facto public domain license, due to its spirit. It's not. The GPL
spells out its terms very clearly, and Oracle certainly has lawyers
that understand the GPL.</p>
<p>Which leads me to my third and final point. Eben Moglen is a
genuis. He's someone that I admire immensely and have at time been
inspired by and aspired to his skills as both a thinker and an
orator. Knowing his history and feeling as I do about him, this post
seems entirely out of character. So then if it's out of character, why
would he do it? Why would he side with a large, powerful corporation
over GPL-using kernel developers? The only conclusion that I can come
to is that he must be recieving money from Oracle. I'd love to be able
to either support or negate this but the Software Freedom Law Center
does not publish the names of the companies that donate to it. In
other words, we can't know what corporate interests have donated to
SFLC because SFLC had decided to not disclose its corporate donors.</p>The Rule of One2015-03-31T13:44:10-04:002015-03-31T13:44:10-04:00Serge Wroclawskitag:blog.emacsen.net,2015-03-31:/blog/2015/03/31/the-rule-of-one/<p>If you care about Free and Open Source Software, you probably think
about ways to support the people who write the programs that you rely
on. In this post, I'm going to tell you what I do, and what I hope
others will do too. I call it the <em>The …</em></p><p>If you care about Free and Open Source Software, you probably think
about ways to support the people who write the programs that you rely
on. In this post, I'm going to tell you what I do, and what I hope
others will do too. I call it the <em>The Rule of One</em></p>
<p>The Rule of One is very simple:</p>
<p><q>Whatever you spend on proprietary software, donate that same amount to Free Software causes.</q></p>
<p>If you're well off, you should employ the <em>Rule of Two to One</em></p>
<p><q>Whatever you spend on proprietary software, donate double to Free Software causes.</q></p>
<p>Think about all the times that we buy proprietary software:</p>
<ul>
<li>
<p>Did you buy a game for your smart phone recently? Or for that
matter, did you buy a smart phone that came with proprietary software
on it?</p>
</li>
<li>
<p>Did you buy some proprietary hardware, like a Roku , or maybe a
"Smart" television? Or even a medical device?</p>
</li>
<li>
<p>Did you buy a video game or game console system?</p>
</li>
</ul>
<p>If you consider all the times that we either purchase or subsidize
proprietary software, for many of us, it ends up being hundreds,
possibly thousands of dollars a year.</p>
<p>I'm not telling you to stop buying the things you love. Instead, I'm
saying that whatever you're spending on proprietary software, invest
in creating the Free alternatives.</p>
<p>If you're not sure how, most of the larger projects have a Donate
button on their home page. Alternatively, here are some organizations
that support Free Software that you should consider donating to:</p>
<ul>
<li>
<p><a href="http://sfconservancy.org/">Software Freedom Conservancy</a></p>
</li>
<li>
<p><a href="http://fsf.org">The Free Software Foundation</a></p>
</li>
<li>
<p><a href="http://www.spi-inc.org/>"Software in the Public Interest</a></p>
</li>
<li>
<p><a href="https://www.gnome.org/foundation/">The GNOME Foundation</a></p>
</li>
<li>
<p><a href="https://ev.kde.org/">KDE e.V.</a></p>
</li>
<li>
<p><a href="https://www.mozilla.org/en-US/foundation/">The Mozilla Foundation</a></p>
</li>
</ul>NYC Blooms with OpenStreetMap2014-12-01T06:54:16-05:002014-12-01T06:54:16-05:00Serge Wroclawskitag:blog.emacsen.net,2014-12-01:/blog/2014/12/01/nyc-blooms-with-openstreetmap/<p>A community mapping project in New York is showing how OpenStreetMap
can not only show how mapping can help people navigate, it can shape
how people perceive their city.</p>
<!--more-->
<p><img src="/images/community_garden.png" height="422" width="380" align="left" hspace="100" /></p>
<p>New York City contains hundreds of
<a href="http://en.wikipedia.org/wiki/Community_gardening">community gardens</a>. These
gardens represent the the hard work of thousands of New Yorkers coming …</p><p>A community mapping project in New York is showing how OpenStreetMap
can not only show how mapping can help people navigate, it can shape
how people perceive their city.</p>
<!--more-->
<p><img src="/images/community_garden.png" height="422" width="380" align="left" hspace="100" /></p>
<p>New York City contains hundreds of
<a href="http://en.wikipedia.org/wiki/Community_gardening">community gardens</a>. These
gardens represent the the hard work of thousands of New Yorkers coming
together to make these spaces come to life, and in some cases growing
their own vegetables to be either given away or sold at local
markets. Yet these gardens don't appear on most maps. Only a few of
them appear on Google Maps, and none of them appear on the New York
City maps, where the community gardens appear as vacant lots. </p>
<p><a href="http://www.meetup.com/osm-nyc/">OpenStreetMap NYC</a>, the local
OpenStreetMap community group, in collaboration with
<a href="http://www.grownyc.org/">GrowNYC</a>, a New York City based
environmental non-profit, are working to increase the visibility of
these gardens through a collaborative mapping project. GrowNYC has
released their data about New York community gardens to the
OpenStreetMap community to help encourage their inclusion on the
map. Because of OpenStreetMap's flexible way of describing features,
mappers are able to capture not only the name and address of a
community gardens, but also features like the hours that the garden is
open to the public, and even whether or not the garden offers
composting.</p>
<p>On November 22nd, OpenStreetMap NYC kicked off this collaboration
through a meetup where local community members came together to work
on mapping the gardens. Leading this meetup was Eric Brelsford, a NYC
Mapper and founder of <a href="http://596acres.org">596 Acres</a>, a non-profit
dedicated to land use and advocacy issues in New York City and Mara
Gittleman from the
<a href="http://nyccgc.org/">NYC Community Garden Coalition</a>, an activist
organization working to protect community gardens in the city.</p>
<p>This collaboration presents benefits to both GrowNYC and
OpenStreetMap. As Mara Gittleman explains, "Having an up-to-date map
can help gardeners, researchers, and allies make the case for
community gardens to funders and policy makers by showing their
proximity to things like schools, illustrate where they might make up
for lack of access to parks, quantify environmental benefits, etc."
Unlike proprietary maps like those from Google, the maps provided by
OpenStreetMap give direct access to the geographic data, allowing for
all the benefits to researchers and policymakers.</p>
<p>Using OpenStreetMap also presents benefits for to the community
gardens. Garden organizers and maintainers can now keep the
information about their gardens up to date on their own, without
needing an intermediary. And because OpenStreetMap is used by so many
websites, these community gardens will be getting exposure to millions
more people than they have before.</p>
<p>Because of its collaborative nature, its purposefully neutral stance
on data and its extensible data representation, it seems likely that
more community groups will use OpenStreetMap to shine a spotlight on
previously invisible features of our world.</p>The Maintenance of Imported Data in OpenStreetMap2014-03-13T09:10:06-04:002014-03-13T09:10:06-04:00Serge Wroclawskitag:blog.emacsen.net,2014-03-13:/blog/2014/03/13/the-maintenance-of-imported-data-in-openstreetmap/<p>Some feedback about my <a href="http://blog.emacsen.net/blog/2014/01/25/why-imports-in-openstreetmap-are-controversial/">previous import post</a>
was regarding my contention that imported data is harder to
maintain. This conclusion is based on years of observation. In this
post, I'll explain why I believe imported data is not the same as
manually mapped data in terms of maintenance.</p>
<!--more-->
<p>The standard …</p><p>Some feedback about my <a href="http://blog.emacsen.net/blog/2014/01/25/why-imports-in-openstreetmap-are-controversial/">previous import post</a>
was regarding my contention that imported data is harder to
maintain. This conclusion is based on years of observation. In this
post, I'll explain why I believe imported data is not the same as
manually mapped data in terms of maintenance.</p>
<!--more-->
<p>The standard disclaimer applies here. I am speaking for myself and not
representing any other group. In addition I want to say that this post
is not meant to discourage imports in general, but that if
OpenStreetMap is to have imports, we need to have comprehensive
discussions about the issues around them.</p>
<p><strong>Adding vs Correcting</strong></p>
<p>The most common issue cited about why imported data is harder to work
with is that it's easier to fill in an empty map. This often leads to
the debate about completeness vs accuracy- in other words is it better
to have a map that is full of partially accurate data or one that is
only partially filled but with completely accurate data? This debate
misses the real issue, which is the relative difficulty of adding vs
correcting OpenStreetMap data.</p>
<p>On a map that is empty, it's easy to see what needs to be added. If a
road or building is not present they can be added. We see this pattern
being played out in OSM as larger, more prominent roads get filled in
first, followed by secondary roads, and finally residential roads,
buildings, POIs, etc.</p>
<p>The more difficult task is determing what data needs correction on a
map that is visually complete. Looking at this map of Washington, DC,
we see roads, buildings, and a variety of POIs. This map may be out of
date. The speed limits on the roads may have changed, a building may
have been torn down, or a store may have changed names.</p>
<p><img src="{{ root_url }}/images/dc-map.png" /></p>
<p>It's not possible to determine what information needs correcting
simply by looking at the map, nor is it always possible by looking at
aerial imagery. Updating the map is far more difficult than creating
it. Either the mapper must know the OSM data is wrong and re-survey
the area, or they are going off newly collected data and the new data
must be reconciled with the old data. For example we might know that a
store is present on a particular street, but to be accurate, we must
know not only that the new store exists, but that it replaced the old
store in the same location.</p>
<p>When data is imported, the map appears complete, and the burden of the
community effort shifts from collection to maintenance, which is a task
that most mappers are less familiar with. In addition, correcting data
does not have the same <a href="http://weait.com/i-want-to-see-it">intrinsic motivation</a>
as first time collection does.</p>
<p><strong>Complex Objects</strong></p>
<p>The second issue that imports sometimes bring about are complex
objects that are difficult or impossible to fix. </p>
<p>In this image, we see a section of New Jersey in OpenStreetMap, with
the patchwork of color representing landuse data</p>
<p><img src="{{ root_url }}/images/nj-map.png" /></p>
<p>When we zoom in, we see one of these objects is a multipolygon
relation, an object composed of several other objects. This
one object references five other objects. In addition, the outer
boundary of this woods shares geometry with about another ten landuse
objects. </p>
<p>If someone were to try to update the landuse data, they might
unintentionally be effecting up to a half dozen other objects at the
same time, meaning that a simple update process now becomes a
difficult and time consuming process of ungluing multipolygons,
splitting them and trying to reconstruct them.</p>
<p>For an experienced mapper, this would be a tedious, time consuming
process. For a new mapper, it would be an daunting and possibly
confusing challenge. Most mappers simply avoid the work altogether and
the data is never updated.</p>
<p><strong>Unverifiable Objects and External Unique Identifiers</strong></p>
<p>The third, and in my view the most common problem we encounter with
imported data is the issue around external identifiers in the imported
dataset.</p>
<p>Let's start with a simple example of the problem in action: New York
City bike racks.</p>
<p><img src="{{ root_url }}/images/bike-racks.png" /></p>
<p>As you can see, there are two bike racks on this street according to
the data.</p>
<p>If one of those racks were found to be missing or in the wrong
location, it would need to be removed. The problem is that it's not
obvious by looking at a bike rack what bike rack is what. What, then
is the solution?</p>
<p>Taking another example, we see this building footprint from New York
City:</p>
<p><img src="{{ root_url }}/images/buildings.png" /></p>
<p>Every imported building in New York has a Building Identification
Number, or BIN. In OpenStreetMap we've separated out the
garage from the building, whereas the city hasn't. When merging the
city data with data from local mappers, do we apply the BIN to the
garage? To the main building? To neither?</p>
<p>A more common example is data imported from GNIS. GNIS data was
imported many years ago and has many known issues associated with it,
including but not exclusively that many of the GNIS points are far
from the actual location of the object, in some cases a half a mile
away.</p>
<p>Taking a look at
<a href="http://www.openstreetmap.org/node/357581016/history#map=5/34.994/-93.384">this</a>
node, just a few weeks ago, this had a GNIS datapoint for a
school. <a href="http://www.openstreetmap.org/user/Korzun">Korzun</a>, a very
active mapper, visited the location himself and found out it was a
drug rehab center, but was too concerned about the integrity of the
GNIS import to remove the dataset, even though he knew it to be
inaccurate.</p>
<p>If an experienced mapper like him is confused, imagine the confusion
of less experienced mappers?</p>
<p>The end result in all of these situations is out of date, or entirely
inaccurate data in OpenStreetMap.</p>
<p>To address these issues, I have three practical suggestions.</p>
<ol>
<li><em>We should reduce or eliminate the use of external tags, including
identifiers during imports</em></li>
</ol>
<p>For years, importers have been bringing in external data along with
the imported data. This data might include collection dates, unique
identifiers, etc.</p>
<p>As part of the import review process, OSM has generally been
reducing the import of much of this data (for example, collection
dates), but we continue to import external identifiers. The idea
behind including the external identifiers is that it's hoped that
at some future time, we'll be able to do an update based on a new
dataset and use the external identifier as part of that process.</p>
<p>Unfortunately, while this possibility has been discussed a number
of times, in the nearly ten years of the project, no one has
successfully been able to make this happen. The reason is that even
if an external identifier is consistent across revisions (which is
not always the case), OSM activity does not lend itself well to
using the external ID as the only identifier in the merge process.</p>
<p>OSMers may decide to add features that are not present on the map
between imports, or may decide that a feature which has a single
identifier in the external dataset should be represented in OSM by
more than one object (eg splitting a road, or separating two
buildings connected by a skybridge). It's even possible that a
mapper will accidentally modify a tag that they do not
understand. This was quite common when <a href="http://openstreetmap.us/2013/05/road-not-rd/">bot-mode</a>
expanded names across the US. Many times it was unable to fix a
road simply because the TIGER tags had been edited by a mapper who
did not understand them.</p>
<p>Because of this, merges must take other factors in consideration
when doing a merge, including the feature name and location. While
slower, these methods prove superior during a merge- leading back
to the original question of why use the unique identifier in the
first place if it creates confusion (leading to stale data) and
does not offer benefits in the long run.</p>
<ol>
<li><em>Imported data must always be imported by hand, and not by script</em></li>
</ol>
<p>When imports began, the process for getting the external data into
OSM was that the data was converted into OSM and then simply placed
into OSM. As we discussed earlier, it is easier to add good data
than correct bad data so we should take advantage of this in future
imports.</p>
<p>We have tools to make this easier, including the <a href="http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager">OSM Tasking Manager</a>,
which is being used in such an import now. Updating OSM this way
not only gives an import more review, but can also be used a means
of engaging users, as was shown in the Seattle import.</p>
<ol>
<li><em>An import proposal must be accompanied by a means of updating the
data in the future</em></li>
</ol>
<p>OpenStreetMap is only valuable if the data on it is not only
complete, but current, and we've seen through OSM that data from
imports is not often updated. We need to change this, and the best
way to do it would be to require that any import specifically
address future updates.</p>
<p>If possible, that process will not only involve conflating the data
from the external dataset with OSM, but continuing to keep OSM
contributors engaged in OSM editing.</p>
<p><strong>Conclusion</strong></p>
<p>Imports have their place in OSM, but they are not without their
perils. I've tried to outline specifically how the past and current
import process does not serve either data consumers nor the OSM
community in general, as well as provided concrete suggestions on
moving forward. I hope that we can continue to make OSM not only the
freest, but the best geographic dataset in the world.</p>Esperanto After Four Weeks2014-02-13T09:38:48-05:002014-02-13T09:38:48-05:00Serge Wroclawskitag:blog.emacsen.net,2014-02-13:/blog/2014/02/13/esperanto-after-four-weeks/<p>I'm into my fourth week of Esperanto and I thought I'd write some
reflections on the process, the struggles and successes.</p>
<!--more-->
<p><strong>The Methods</strong></p>
<p>My main method of learning Esperanto is the site
<a href="http://lernu.net">lernu.net</a> and its "Bildoj kaj demandoj" (Pictures
and Questions) course. In addition, I've gotten some help from …</p><p>I'm into my fourth week of Esperanto and I thought I'd write some
reflections on the process, the struggles and successes.</p>
<!--more-->
<p><strong>The Methods</strong></p>
<p>My main method of learning Esperanto is the site
<a href="http://lernu.net">lernu.net</a> and its "Bildoj kaj demandoj" (Pictures
and Questions) course. In addition, I've gotten some help from
Esperanto speakers, and I've been listening to Esperanto music from
<a href="http://muzaiko.info/">Muzaiko</a>, as well as the
<a href="https://www.esperanto-usa.org/en/node/551">Esperanto-USA</a> podcast.</p>
<p>Lastly, when specific questions arise, I've been using the <a href="http://donh.best.vwh.net/Esperanto/rules.html">16 Rules
of Esperanto Grammar</a>,
the <a href="http://www.gutenberg.org/ebooks/16967?msg=welcome_stranger">English-Esperanto Dictionary</a>,
and on occasion, simply using Google Translate.</p>
<p><strong>Quantity</strong></p>
<p>I don't study Esperanto every day, but I do lessons about three or
four times a week, at least. The amount of time I spent depends on how
quickly I feel I'm absorbing the lessons. If I'm understanding the
lesson well, I do more, and if I'm having trouble, I go slower.</p>
<p><strong>Process</strong></p>
<p>For me one of the more illuminating experiences has been in seeing how
my own mind handles the new material. Esperanto is quite easy, but
occasionally I find myself having difficultly with the
material. Lernu has been an incredible resource for that. Using Lernu,
I'm able to set my own pace, learn as much as I can, and then put the
computer away. When I'm ready to take on the next lesson, it's there
for me.</p>
<p>In addition, after each section, the software gives small
quizzes. Each quiz question covers material in the previous lesson. If
you struggle with a question, the software offers you the opportunity
to see various ways the question can be answered.</p>
<p>In order to get the feel of the spoken language, I listen to Esperanto
music and spoken Esperanto in the podcasts. In order to get the
"mouth feel", I try to speak every section out as I go along in the
lessons. Lernu provides audio for every question/answer pair, which
makes it easy to repeat as I go along.</p>
<p><strong>Results</strong></p>
<p>I've worked my way through about two thirds of the basic language
course.</p>
<p>In just four weeks of learning Esperanto, I can pick up on about 20%
of the Esperanto USA podcast. It's not enough to get the gist of the
conversation, but it's enough to pick up meaning here and there. In
terms of reading, I'm at about the same level. I can figure more from
context and the use of a dictionary, but it's very slow.</p>
<p>On the positive side, I feel that I've been able to learn at a very
steady pace and I feel comfortable learning more, which is good
because there's a lot more to learn.</p>
<p><strong>Meta-learning</strong></p>
<p>In addition to just learning Esperanto, I've learned a lot about how I
learn languages, which was a big part of this experiment.</p>
<p>Spelling is a major issue for me, even in English, and Esperanto
hasn't been an exception. Even though Espernato is entirely phonetic,
I often make simple mistakes, spelling "ĉi" as "ĉe", or "li" as "le",
or transposing two letters.</p>
<p>Vocabulary has been a mixed bag. Many of the words in Esperanto share
a root either with English or Latin, making them easy to
remember. Some words have no equivalent, and those words require work
to memorize. A technique that I've found very helpful is to use either
a mnemonic device to help me memorize a word until the word can "stand
on its own". For example the word for "when" is "Kiam", and I
remembered that by knowing the root "ki"- followed by "am", which
reminds me of "AM/PM". I was struggling with "kvar", the number four,
and so I visualized the band <a href="http://en.wikipedia.org/wiki/Gwar">Gwar</a>
and imagined that there are four members. The fact that there are
actually five members of Gwar is unimportant, since it's the
connection that I'm trying to build. For the word "mateno" (morning),
I associated it with the English word "maternity", which I associate
with birth, visualizing the day being born. And lastly, I remember the
word "pilko" (ball) by thinking of
<a href="http://media.joe.ie/wp-content/uploads/2013/09/karl2.jpg">Karl Pilkington</a>.</p>
<p>The Lernu site has tests, and a majority of the questions that I've
gotten wrong have been spelling errors, followed by forgetting the
correct tense of a word, followed by forgetting a word
altogether. This mirrors my experience in English, where even today, I
misspell words or forget a word in the middle of a sentence
occasionally. </p>
<p><strong>Conclusion After Four Weeks</strong></p>
<p>I've been really happy with this experiment, both in learning
Esperanto, and learning about my own learning process. I think the
Esperanto is going at a speed I can expect based on the time I put
into it, and as I go, I'm learning an enormous amount about myself in
language acquisition.</p>Why Imports in OpenStreetMap Are Controversial2014-01-25T21:05:35-05:002014-01-25T21:05:35-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-25:/blog/2014/01/25/why-imports-in-openstreetmap-are-controversial/<p>OpenStreetMap's goal is to map the entire world, so one might assume
that anything that would help give the project a leg up would be
welcome, but what many potential importers find is that the OSM
community, in particular the more senior members, are hostile to
imports. Understanding their strong …</p><p>OpenStreetMap's goal is to map the entire world, so one might assume
that anything that would help give the project a leg up would be
welcome, but what many potential importers find is that the OSM
community, in particular the more senior members, are hostile to
imports. Understanding their strong feelings can be difficult, but I'm
going to take a look at the issues around imports in the community,
and why they're so controversial.</p>
<!--more-->
<p>Before I go on, I need to make clear that the views in this post are
my own and don't reflect the views of the DWG, the OSMF, OSM US or any
other organization.</p>
<p><strong>OSM's Construction</strong></p>
<p>OpenStreetMap is a single unified geographic dataset. As a
consequence of its design, it's far more integrated than even
professional GIS maps are. For example, you can't simply remove all
the roads objects in OSM, because those road objects might be
connected to other objects, such as political boundaries, or natural
features. Most professional GIS maps use a system of layers which are
placed on top of one another. In contrast, OSM maps are more like a
complex weaved material, with geographic features intertwined with
each other.</p>
<p>This makes some operations in OpenSteetMap very easy. If a political
boundary runs along a river, for example, as the river becomes more
detailed, so does the political boundary. At the same time, it poses a
set of challenges for imports.</p>
<p><strong>Licenses</strong></p>
<p>The most common problem with a data import into OpenStreetMap are
licensing conflicts. In an ideal world government datasets would be
released into the public domain. Unfortunately this is the exception
rather than the norm. In the United States most states and
municipalities do not release their geographic datasets to the public
without restrictive licenses and expensive license fees.</p>
<p>Some municipalities are trying to make their data more available but
do not use standard licenses. Instead, most US municipalities
create their own licenses from scratch, oftentimes with conflicting
terms. It's not uncommon for a municipal website to say that the data
is in the public domain in one part of the website, but then another
part of the website will say that the data is under copyright and may
not be used or distributed without explicit permission.</p>
<p>The situation in the US mirrors much of the rest of the world, where
national or local governments struggle between "openness" and
"control".</p>
<p>The result for OpenStreetMap is that getting a license situation clear
is difficult or sometimes impossible. Because of the way OSM data is
so heavily integrated, removing it afterwards is nearly impossible, so
the project needs to be extremely careful beforehand about the license
situation for any imported data.</p>
<p><strong>Official vs Authoritative</strong></p>
<p>The second challenge or imports is the issue of official vs
authoritative data. Most people assume that if a dataset is official
that means that the data is of extremely high quality. This can be
true, but more often than not, the official dataset is not what one
would expect.</p>
<p>When a government creates or maintains a dataset, it does so with a
specific purpose in mind. For example, in the US TIGER dataset, the
purpose of the data is to provide a mapping of roads in order for
census employees to know where to look for houses. The TIGER dataset
needs to be complete enough for the census takers, but that is not the
same as being a comprehensive road network map. For example, if a
house exists at the end of the road, it does not matter to a census
taker if the road has a slight bend in it or not.</p>
<p>Similarly, other data sets are often created to meet a very specific
need. In order to meet that need, a local government will often
outsource the task of surveying, or will hire low wage employees to do
the data collection. The results of the survey need to be consistent,
and governments often provide exhaustive procedures and checklists, but
only for the specific objective they're trying to meet.</p>
<p>This difference in mapping objectives often comes out in address
data. A government dataset may provide addresses as points on a map
but it's often unclear what these points correspond with. Do they
correspond with the geometric center of a building, or do they
correspond with an entrance? If they correspond with an entrance, what
do they do in cases where the building has multiple entrances?</p>
<p>In practice we find that such datasets are often complete in that they
show all the addresses, but are inconsistent in where the address
points are placed across buildings.</p>
<p>This issue of accuracy and consistency becomes more pronounced on
larger, national datasets. A national dataset often consists of a
collection of regional datasets, each which have been collected by
different individuals and organizations, leading to inconsistency.
Paradoxically the national datasets are the ones that OpenStreetMap
members are most interested in, since they can provide the most
information.</p>
<p><strong>Doing it Right</strong></p>
<p>Once the issues of license and data quality are addressed, there are
questions of how to get the data into OSM. As mentioned earlier, one
can't simply add external data onto OSM- the data needs to be
integrated.</p>
<p>This integration consists of many highly technical, highly detail
oriented steps. Every new importer believes that the integration step
will be easy, but the fact is that the process is difficult and
tedious.</p>
<p>OpenStreetMap is full of very badly imported data, and suffers from it
to this day.</p>
<p><strong>Corrections and Updates</strong></p>
<p>Unfortunately, even after nearly a decade, no one has found an ideal
solution to this problem. The result is usually that data imported
into OSM never gets updated and the imported data in OSM becomes wrong
very quickly.</p>
<p>When considering updates, as discussed previously, we cannot assume
that an administrative database is correct. Ideally we want OSMers to
map the areas themselves, and it's inevitable that they will do so,
leading to a conflict between the two datasets.</p>
<p>OSMers have oscillated between trying to update data based on
government datasets and always assuming that local changes are
correct. Neither of these techniques will work in every case. Instead,
a mapper will need to check the conflict manually, preferably by
manually surveying the area.</p>
<p>This process of updating is slow and tedious and has only happened in a
handful of cases.</p>
<p><strong>Community Helping or Harming</strong></p>
<p>Within OpenStreetMap there is a debate on whether or not imports help
or harm the OpenStreetMap community.</p>
<p>OpenStreetMap works because its members are constantly adding and
correcting information in the system. If there were no community
members collecting this data, the map would quickly suffer bitrot,
where old data sticks around and is not replaced with current, up to
date geographic information.</p>
<p>Because of that, some members believe that efforts should be focused
not on bringing data in from external sources, but on growing the
community of mappers. Others believe that "seed data" provides a
starting point for more mapping, especially in places that have not
been very active in OpenStreetMap previously.</p>
<p>I'm not going to rehash these arguments here, but I will say that the
role of imports in OpenStreetMap remains contentious, and an area of
research that I would like to see explored is the rate of update
between manually surveyed and imported data.</p>
<p><strong>Conclusion</strong></p>
<p>OpenStreetMap only works because of our community of many thousands
of dedicated mappers who spend their time updating our map. Whatever
the role of imports today or in the future, they can never be a
substitute for on the ground mapping, which needs to remain a
cornerstone of our efforts in OpenStreetMap.</p>Fearing French and Embracing Esperanto2014-01-23T14:01:10-05:002014-01-23T14:01:10-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-23:/blog/2014/01/23/fearing-french-and-embracing-esperanto/<p>This post is about why I fear French, and what my attempt at taking
this problem on has taught me. This is one of the most personal blog
posts I've made in several years, but in it are lessons for nearly
everyone, especially if you are considering learning a second …</p><p>This post is about why I fear French, and what my attempt at taking
this problem on has taught me. This is one of the most personal blog
posts I've made in several years, but in it are lessons for nearly
everyone, especially if you are considering learning a second language</p>
<!--more-->
<p><strong>[UPDATE 01-25-2014]</strong>: There seems to be some confusion about my use of the term
"polyglot" in this post. I am not equating programming languages with
spoken languages. I am comparing the introduction of a new skill in an
existing domain. Also, I'm referencing a talk by Adam Jacob, a well
known member of the system adminstrator community about how we system
administrators are "polyglot programmers".</p>
<p><strong>My Background</strong></p>
<p>I was born just outside of Paris in 1978. My father was French, and my
mother is an American who went to Paris to perfect her art and ended
up marrying a Frenchman. They intended to spend the rest of their
lives in Paris and so when I was born, they decided to make English
the "home language"- thereby ensuring my fluency.</p>
<p>When I was just over two years old, my parents moved to the United
States, and while I had been absorbing French, I wasn't any more
fluent than any other two year old. English remained our home language
in the US.</p>
<p>French was spoken in my home, but it was never spoken to me. My
parents used it to communicate with each other as a sort of secret
code. They did this in delicate situations outside, but they also used
it to argue in front of me so that I couldn't understand them.</p>
<p>The attitude my parents took was that I should learn French- but that
I should do so on my own. My father recorded <a href="https://en.wikipedia.org/wiki/French_in_Action">French in Action</a> off the
television for me and asked me to watch it.</p>
<p>Every few years, I would return to France with my family and be
reminded by my grandparents and uncle that I should speak French, and
that it was a shame, or a failing, that I didn't.</p>
<p>I took a French class in college, and it also used French in Action,
and I learned only marginally more than I had before, from watching
the course on television- which is very little. While I could
understand the lessons, the structure of the quizzes and the time
allotted meant that I had virtually no time to figure it out. I'd
misread or misunderstand a question, get the first five answers wrong,
then I'd figure it out and get most of the next ones correct. But the
professor never took that into consideration, and I failed each
quiz. Each failure meant each time, I was increasingly deflated, and
eventually I fell behind. Even if I had aced the final exam, I would
have failed the course due to the quizzes.</p>
<p>What makes this tragic is that I am, in fact, a French citizen.</p>
<p>And as a French citizen, I'm required to fill out paperwork in
French. Years of anxiety and shame have left me with a feeling of
dread on the topic. Even writing this out, I feel my jaw clench and
the tips of my fingers become numb.</p>
<p><strong>The Polyglot</strong></p>
<p>Switching gears, as <a href="https://twitter.com/adamhjk">Adam Jacob</a> would
say, I am a polyglot. I do not speak any other natural
language, but I able to program competently in four programming
languages, as well as another half dozen with some help.</p>
<p>In addition to programming, I was a system administrator for over a
decade, where it was my job to work on vastly different systems and in
some cases make them communicate with each other.</p>
<p>How could it be that I could excel at quickly understanding and
working with computer languages and systems, but with spoken language
be nearly crippled. Certainly this was indicative of something being
wrong.</p>
<p>I needed to figure out what the connection, or more accurately the
disconnection was between these two skills.</p>
<p>Mastering a spoken language and programming a computer are entirely
different, but the essence of my search is not comparing the two, but
rather comparing learning an additional programming language with
learning an additional spoken language.</p>
<p>The programming languages people may be familiar with are things like
C, C++, Java or Python. All of those languages are popular, and all of
them are similar. But there are other programming languages too, such
as Javascript, Clojure, Haskell, Erlang and Prolog. You may have heard
of Javascript, but it's less likely that you're familiar with the
others. Programming in any of these languages is different than
programming in Java. Each requires learning not just a syntax, but a
change in programming style altogether. Each of these languages will
teach you a new way of interacting with the computer and a new way of
thinking about computer programming.</p>
<p>Of the languages listed, I only know Clojure and Javascript, but I'm
confident that I could learn any of the languages I mentioned because
I've made the leap in the past. I've learned new programming languages
successfully, even when they required a major change in mindset.</p>
<p><strong>Rethinking Language Learning</strong></p>
<p>How could I transfer my computer language skills to spoken language?</p>
<p>There were lots of TED talks on this very subject.</p>
<p>Chris Lonsdale says <a href="http://www.youtube.com/watch?v=d0yGdNEWdn0">you can be fluent any language in six months</a>, and Conor Quinn
says I can <a href="http://www.youtube.com/watch?v=6a6vVIdQBd0">start speaking in 10 days</a>.</p>
<p>Benny Lewis talks about
<a href="http://www.youtube.com/watch?v=0x2_kWRB8-A">hacking language learning</a>
and Sid Efromvich has <a href="http://www.youtube.com/watch?v=-WLHr1_EVtQ">5 techniques to speak any language</a>.</p>
<p>If you're interested in language acquisition, I suggest watching all of
these talks, as they each teach something different and complementary,
but what they had in common was:</p>
<ol>
<li>
<p>Communicate as early as you can, even if you aren't speaking</p>
</li>
<li>
<p>Don't be afraid to make mistakes, because it's inevitable</p>
</li>
<li>
<p>Learn/speak in a way that's relevant to you personally</p>
</li>
<li>
<p>Retrain your ears and your mouth</p>
</li>
<li>
<p>Language acquisition is a skill</p>
</li>
<li>
<p>Language acquisition is a skill that's best learned when we're
mentally at ease</p>
</li>
</ol>
<p>For me, those last two were the most important.</p>
<p>Knowing that language acquisition itself is a skill changes the
process from "Learning French" to "Learning Language Acquisition", and
all of those years of shaming, whether by family or academia, helps
explain why that process has been so difficult.</p>
<p>There were two ways to approach this problem. The first was to try to
emotionally move past the issues I've had with French and try again,
the other option was to learn an entirely different language in order
to gain the skills of language acquisition and then apply those skills
to French. As you might guess from the title of this post, I opted for
the second and decided to learn Esperanto.</p>
<p><strong>Esperanto</strong></p>
<p>The natural question at this point is either "Why Esperanto?" or
possibly "What is Esperanto?"</p>
<p>Esperanto is a <a href="http://en.wikipedia.org/wiki/Constructed_language">constructed language</a>, in other
words it's a language which was created by a person or persons
intentionally, rather than having developed naturally over hundreds or
thousands of years as most languages do.</p>
<p>I'd known <em>about</em> Esperanto for many years. It's mentioned casually in
the BBC show Red Dwarf, where all the <a href="http://reddwarf.wikia.com/wiki/Esperanto">ship's signs</a> are posted in both English and Esperanto. I'd also heard about it because more than a few
students from Central and Eastern Europe that I knew spoke about it,
but I didn't think anything of it.</p>
<p>It's only now with the convergence of my interest in learning French
and my reading <em>Land of Invented Languages</em> which has sparked my
interest. As a practical exercises in language acquisition, Esperanto
comes with some very impressive credentials.</p>
<p>The aforementioned Benny Lewis
<a href="http://www.fluentin3months.com/2-weeks-of-esperanto/">claims that just two weeks of Esperanto can get you months ahead in your target language</a>,
Esperanto USA claims that Esperanto is <a href="http://www.esperanto-usa.org/node/77">four times easier to learn</a> than any other language,
and Wikipedia has an <a href="http://en.wikipedia.org/wiki/Propaedeutic_value_of_Esperanto">extensive article</a>
on the foreign language benefits of Esperanto.</p>
<p>This idea is summarized well by this talk by Tim Morley entitled
<a href="http://www.youtube.com/watch?v=8gSAkUOElsg">Teach Esperanto First</a>,
where he describes the process of teaching children Esperanto in order
for them to gain language acquisition skills, applicable in any
language.</p>
<p>This idea of a simplified first language resonated with me, just as I think that Python is a great
<a href="http://blog.emacsen.net/blog/2014/01/03/python-for-adults/">first programming language</a>.</p>
<p>There are many resources for learning Esperanto online, including and
especially <a href="http://lernu.net/">lernu.net.net</a>, which I have been
using. In addition, there are books online, including audiobooks at
<a href="https://librivox.org">LibriVox</a>.</p>
<p><strong>Why Esperanto is Easy</strong></p>
<p>I've been using the Lernu online course for about two weeks now and I
have to say that it's been wonderful. Along with some basic
understanding of the <a href="http://en.wikipedia.org/wiki/Esperanto_grammar">Esperanto grammar</a>, I've been
able to learn an incredible amount in a very short time. The language
is extremely <a href="http://en.wikipedia.org/wiki/Elegance">elegant</a> in that
every word is constructed from a root word and then modified using a
series of simple prefixes or suffixes.</p>
<p>From a vocabulary perspective, Esperanto is also very easy as most of
the words have a Latin, German or English root, and because English is
composed so largely of French and German roots already, it makes
acquisition that much easier. For example the word for dog is <strong>hundo</strong>
(nouns all end in -o), the root being the German "hund", which in
English became "hound", and so on.</p>
<p>Between the partially familiar vocabulary and the simple rule based
word construction, understanding Esperanto is fairly
straightforward. Obviously mastering any language takes time, but
Esperanto gives the learner a step up at each hurdle.</p>
<p>Even the simple words have hints about their meaning in them. The word
<strong>kio</strong> such as in the sentence <em>"Kio estas tio?</em> (What is that?) vs
the word <strong>kiu</strong>, such as in the <em>"Kiu estas tiu?"</em> (Who is
that?). The -o at the end of kio indicates that the question is a
"What", which is the same as the -o ending used by all nouns. The -u
indicates a pronoun question, "Who". But just by remembering the -o
noun rule, one can differentiate the two words.</p>
<p>The phonetics of Esperanto are fairly easy. Each word is spelled
phonetically, with no silent letters, two letter combinations or verb
modifiers. The sounds used in the language are relatively familiar to
English speakers who've listened to European languages and are easy to
pick up.</p>
<p><strong>Esperanto Challenges</strong></p>
<p>There are a few challenges with learning Esperanto. Because it's an
auxiliary language, there are fewer people using it every day for one
on one interactions and so you see less original culture in Esperanto
than you do in a natural language.</p>
<p>In addition, because Esperanto is usually learned as a second
language, it's slightly more difficult to train your ears and your
mouth to the language because each Esperanto speaker has a slightly
different accent based on their linguistic or geographic
background. Even the few <a href="http://en.wikipedia.org/wiki/Native_Esperanto_speakers">native speakers</a>
purportedly have slightly different accents. This is somewhat offset
by the fact that most Esperanto is spoken relatively slowly in the
various recordings that I've heard.</p>
<p>All in all, the downsides issues are minor compared to the ease of
language acquisition that Esperanto provides, and there is a lot more
in Esperanto than in other constructed languages such as such as
Lojban, Ido or Interlingua. </p>
<p>** Learning Esperanto**</p>
<p>If you want to learn Esperanto, I recommend the course I'm taking at
<a href="http://en.lernu.net/kursoj/bkd/index.php">Lernu</a>, followed up by
checking out this <a href="http://esperanto.lodestone.org/ligoj/">amazing resource</a> of links to travel
groups, books and podcasts in Esperanto from the Esperanto club at the University of Rochester.</p>
<p>I have a long way to go until I'm fluent in Esperanto, but I feel that
with a new framework for examining language learning as well as the
experience of Esperanto under my belt, I will be in an excellent place
to not only speak with other Esperanto speakers, but to be able to
tackle my fears of French.</p>
<p>Ĝis la revido!</p>Edit Wars in OpenStreetMap2014-01-17T22:12:07-05:002014-01-17T22:12:07-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-17:/blog/2014/01/17/edit-wars-in-openstreetmap/<p>Reading the feedback from my <a href="http://blog.emacsen.net/blog/2014/01/04/why-the-world-needs-openstreetmap/">Why The World Needs OSM</a> post has been
enlightening. One criticism of OSM that has come up multiple times is
the concern about <a href="http://en.wikipedia.org/wiki/Editor_war">edit wars</a>
like those seen in Wikipedia. It's a topic that I know something
about, both as a long time OSM contributor …</p><p>Reading the feedback from my <a href="http://blog.emacsen.net/blog/2014/01/04/why-the-world-needs-openstreetmap/">Why The World Needs OSM</a> post has been
enlightening. One criticism of OSM that has come up multiple times is
the concern about <a href="http://en.wikipedia.org/wiki/Editor_war">edit wars</a>
like those seen in Wikipedia. It's a topic that I know something
about, both as a long time OSM contributor, and also as a
<a href="http://wiki.openstreetmap.org/wiki/Data_working_group">DWG</a>
member. This post explores how common edit wars are and the
circumstances that surround them.</p>
<!--more-->
<p>Before I go any further on this topic, I want to state for the record
that the opinions expressed in this post are my own and don't reflect
the DWG or the OpenStreetMap Foundation in any way.</p>
<p>OpenStreetMap DWG members operate similarly to <a href="http://en.wikipedia.org/wiki/Wikipedia:Administrators">Wikipedia Administrators</a> - we handle issue of users breaking community standards
and intervene in issues of conflict resolution. We don't enforce any
particular mapping conventions, but we do enforce community
guidelines.</p>
<p><strong>What are the kinds of editing conflicts seen in OSM?</strong></p>
<p>The most common conflict is around the issue of data imports. OSM is
not a collection of information sources, but a single unified dataset
where geographic objects interlock. Because of this, and because of
OSM's emphasis on primary sources and community mapping, imports are
very difficult to do right. In response, OSM has a set of <a href="http://wiki.openstreetmap.org/wiki/Import/Guidelines">import guidelines</a>
which must be followed. When they are not, the DWG steps in and
enforces that the guidelines are met.</p>
<p>The second most common place that the DWG steps in is
<a href="http://wiki.openstreetmap.org/wiki/Vandalism">vandalism</a>. Vandalism
is purposefully editing the map in a way that doesn't reflect the
reality. OSM community members often fix small mistakes by newbies or
small scale vandalism. The DWG often gets called in to fix larger
scale vandalism, usually by drive-by troublemakers.</p>
<p>The third most common type of case that the DWG handles are around
tagging conflicts. Tagging is the process of classifying
objects. OpenStreetMap uses a free form folksonony and sometimes
disputes arise about how to classify certain features. I hesitate to
call this an edit war because it is almost always a single user whose
views differ from those in the rest of the community.</p>
<p><strong>What about actual edit wars?</strong></p>
<p>Actual edit wars do occasionally come up, but they almost always
involve political disputes. One group believes that an area of land
belongs to them, and or has a certain name, while another group
believe that they own the land, and or it has a different name and the
two sides cannot come to consensus. In this special case, the DWG has
special
<a href="http://wiki.osmfoundation.org/w/images/d/d8/DisputedTerritoriesInformation.pdf">policies</a>
laid out from the Foundation in how these delicate situations are
handled.</p>
<p>When an edit war happens, the DWG gets involved, but the frequency of a
genuine edit war is very low, on the order of one edit war every six
months or longer. It's far less frequent than what someone outside the
project may imagine.</p>
<p><strong>Why are edit wars so common in Wikipedia, but so uncommon in
OpenStreetMap?</strong></p>
<p>Wikipedia edit wars concern two issues, facts and presentation. In
particular, Wikipedia relies on <a href="http://en.wikipedia.org/wiki/Wikipedia:Identifying_and_using_primary_and_secondary_sources">secondary sources</a>. When
two facts collide, an editor must decide which source is
better. OpenStreetMap relies instead on primary sources of
contributors visiting an area and editing OSM to reflect what they
see. Secondary sources are considered inferior and many OSMers use the
derogatory term <a href="http://wiki.openstreetmap.org/wiki/Armchair_mapping">armchair mapping</a> in
reference to using these sources. Because of this, when a conflict
arises, the person who visits the location is going to be given
deference over any other source, even officially published government data.</p>
<p>As for presentation, Wikipedians may argue over the best way to write
a paragraph, or where to list a particular fact. In OSM the general
rule is to update wrong information, and to add detail when you
can. </p>
<p>I think that our policies around ground observable truth are the main
reason why edit wars are infrequent and almost always concern
political boundaries, which are the one feature that OSM maps that is
not ground verifiable.</p>
<p>The other reason that I believe edit wars are less common in OSM is
that OSM has a higher bar required for editing. OSM does not allow
anonymous edits, and unlike Wikipedia. If a particular user is causing
problems, they can be easily identified.</p>
<p>Additionally, OSM is quite a bit smaller than Wikipedia. Even though
OSM has over 1.2 million user accounts, on a daily basis only about
2000 users ever edit the map. While 2000 is certainly not a small
number, it's relatively easy to spot a troublemaker. But even then, a
single user cannot start a war- someone who changes large parts of the
map on their own is an isolated vandal, while actual edit wars would
require at least two parties. This virtually never happens.</p>
<p>When another editor makes a change to something I've created, in every
case I've seen, they've enhanced the map. Either they've expanded on
the data that I entered or they corrected a mistake I made. This is
the norm in OpenStreetMap- additive and corrective edits. Edit
conflicts are so infrequent that most users will never encounter them,
and even to the group that oversees them, they're a special
case. It's possible that as OSM grows that the number of edit wars
will increase, but I don't think that's likely, as we haven't seen the
number of edit wars grow despite the project's growing success over
the years. Instead, I think that the source of edit wars, that is the
number of geopolitical conflicts is going to be a better predictor of
the trend.</p>The Role of the Fringe2014-01-14T08:15:27-05:002014-01-14T08:15:27-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-14:/blog/2014/01/14/the-role-of-the-fringe/<p>I'm in the process of reading <em>In The Land of Invented Languages</em>- an
excellent book I will write more about. For this post,
I want to focus on how Esperanto's community is like our Free
Software/Free Culture community and the role of the fringe element.</p>
<!--more-->
<p>Reading about Esperanto and …</p><p>I'm in the process of reading <em>In The Land of Invented Languages</em>- an
excellent book I will write more about. For this post,
I want to focus on how Esperanto's community is like our Free
Software/Free Culture community and the role of the fringe element.</p>
<!--more-->
<p>Reading about Esperanto and the struggles it's had over the years to
be taken seriously in spite of a "fringe element" who are both
exuberant in speech (and dress) as well as philosophically motivated
reminded me of the struggles the Free Software movement. In
fact these two movements are more similar than dissimilar in many
ways. The answer to why this is this touches on the very core of both
communities, which is the social cohesion between people who believe
that they are changing the world. </p>
<p>The Free Software community has without a doubt changed the world. The
Internet as we know it is built on and with Free Software. Billions of
people use Free Software every day, either directly or indirectly, and
Free Software has changed the dialog about the underlying assumptions
our society holds about issues of morals and economics and how those
underlying assumptions shape our actions and our laws, both as
a group and as individuals.</p>
<p>Esperanto speakers set out to change the world by creating a unified
language that cast off previous cultural influences. Esperantians
wanted a neutral language in order to share their culture to the
world using a dialect that wasn't shaped by any particular kind of
thinking.</p>
<p>Both movements have been shaped and proselytized by those espousing
its practical benefits over its philosophical ones. In the Free
Software community, we have the Open Source movement, which has
arguably overshadowed the older Free Software movement. In the
Esperanto community, similar arguments broke out amongst its members.</p>
<p>In both communities, community members and community leaders
complained about "the fringe", those members, often men, who didn't
take their grooming too seriously, who dressed in odd outfits (in the
case of Esperanto, lined with green stars- the Esperanto symbol),
etc. Anyone who has spent a significant amount of time in either
community probably knows what I'm talking about, and have had the same
thoughts along the lines of "How can we be taken seriously with these
people?" If you haven't had that thought, then you're probably one of
the people everyone else is thinking this about.</p>
<p>The answer is tricky. After seventeen years in the Free Software
community, I know without a doubt that these people have scared others
away. In particular they've scared away people who weren't affluent,
white, eccentric males. But on the other hand, they're the ones who
still come to all the events and who speak publicly about the Free
Software movement's philosophical underpinnings.</p>
<p>With any peaceful group that is trying to change the world, maybe
these people are inevitable. The question for the rest of us is how
to grow the community to be large enough that others aren't scared
off, and that everyone has a home.</p>The Scoop on Cat Litter2014-01-12T11:09:35-05:002014-01-12T11:09:35-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-12:/blog/2014/01/12/the-scoop-on-cat-litter/<p>It's time to take a break from the high minded to the mundane. Today
I'm doing a review of cat litter!</p>
<!--more-->
<p>I have two wonderful cats, one male and one female, both about one
year old, both about 7lbs, and both adopted from the
<a href="http://www.aspca.org/nyc">ASPCA</a>.Since I've owned cats as …</p><p>It's time to take a break from the high minded to the mundane. Today
I'm doing a review of cat litter!</p>
<!--more-->
<p>I have two wonderful cats, one male and one female, both about one
year old, both about 7lbs, and both adopted from the
<a href="http://www.aspca.org/nyc">ASPCA</a>.Since I've owned cats as an adult,
I've been searching for the perfect cat litter. I've tried six
different litters so far, with varying results.</p>
<p>My regime is to scoop the cat box once a day, and replace the litter
about once a week, adding more cat litter as necessary to replace
what's been used up. I have a very large <a href="http://www.natures-miracle.com/products/cat-litter-boxes/hooded-litter-box-with-cover-for-multiple-cats.aspx">cat box</a>,
which can easily hold 20lbs of clay litter.</p>
<p>Living in Manhattan, there is a premium on space, and so a cat litter
needs to do excellent odor control, more than it might if I lived in a
larger space and could put the cat box in a more out-of-the-way spot.</p>
<h1>Feline Pine</h1>
<p>The first cat litter up is the <a href="http://www.felinepine.com/">Feline Pine</a>. This is what the local ASPCA uses,
so I thought it would be a good idea to use the same product during
the cats' transition. This is a dust free, chemical/fragrance-free
product made from wood.</p>
<p><strong>Pros:</strong> I like that this is an all natural product, made from a
renewable resource. It also scooped/sifted well.</p>
<p><strong>Cons:</strong> The product smelled a bit like used cat litter (ammonia) to
me even when just out of the bag. I thought the smell would
dissipate but it just saturated the apartment and I had to get rid
of it.</p>
<p>It also didn't do any clumping, although the company apparently also
makes a <a href="http://www.felinepine.com/products/feline-pine-clumping-litter.aspx">clumping version</a>.</p>
<p><strong>Overall:</strong> 2 out of 5. Of all the products, this one irritated me
the most even when brand new. I like the idea that it's all natural,
but ultimately the product needs to be something I can live with.</p>
<h1>World's Best Cat Litter</h1>
<p>The next product on my list is <a href="http://www.worldsbestcatlitter.com/">World's Best Cat Litter</a>, another all natural
(corn based) product. I've met several people who use World's Best and
it seems to have a good reputation.</p>
<p><strong>Pros:</strong> The biggest advantage to World's Best is that it's
flushable. You can simply scoop, sift, and toss out kitty waste along
with your own. It won't clog or damage your pipes. It's also all
natural, uses no artificial chemicals or perfumes. It also claims to
be longer lasting than other products.</p>
<p><strong>Cons:</strong> Firstly, and most importantly, I was unhappy with the odor
control. After a few days, the ammonia from the cat box smelled
throughout the apartment. Secondly, the litter tracked throughout the
entire apartment and were a mess to find/clean up. Thirdly, the urine
clumps sometimes crumbled on their way to the toilet.</p>
<p><strong>Overall:</strong> 2 out of 5. I wanted very much to like this product, but it
disappointed me. It worked better than Feline Pine, and it is
environmentally friendly, but the odor control and tracking issues
need addressing. </p>
<h1>Yesterday's News</h1>
<p>The third product on the list is <a href="http://www.yesterdaysnews.com/">Yesterday's News</a>. This product is different than
the other products in that it's made from recycled newspapers, formed
in pellets, rather than in small granules like the other litters.</p>
<p><strong>Pros:</strong> This product uses recycled newspaper, which is good
environmentally. It's also dust-free, just like the World's Best and
Feline Pine. It's also completely odorless.</p>
<p><strong>Cons:</strong> This product does a decent job of odor control for a few
days, but after that, the urine soaked newspaper pellets at the bottom
of the litter lose their appeal quickly. The product is also not
siftable- the pellets are too large. One of my cats cries when he
walks on the product, and after accidentally stepping on a few stray
pellets in the apartment, I cry too.</p>
<p><strong>Overall:</strong> 2 out of 5. I love the idea of using recycled newspaper,
and it does provide some odor control, but not nearly enough to make
up for its inability to be scooped and sifted.</p>
<h1>Fresh Step Unscented Clay Litter</h1>
<p>The online reviews suggested that Fresh Step is the best cat litter
available, but people complain about the perfume, saying it has a
chemical odor. I decided to try the Fresh Step Unscented Litter.</p>
<p><strong>Pros:</strong> Odor control at last! Of all the cat litters I've tried,
this one does the absolute best in controlling odor. Even when
cleaning/sifting, it reduces the odor to a very controllable
level. It's easily siftable, it clumps fairly well, and it doesn't
track as much as some of the other litters.</p>
<p><strong>Cons:</strong> Firstly it should be noted that clay cat litter is mined,
and is generally not considered
<a href="http://www.greenlivingtips.com/articles/Cat-litter-and-strip-mining.html">environmentally friendly</a>. On
a personal level, this product is particularly dusty- the most dusty
of all the products I've used. Pouring it out or sifting it releases a
cloud of cat-litter dust into the air each and every
time. Additionally, it does have a tendency to clump on the bottom of
the pan and need to be scooped up forcibly.</p>
<p><strong>Overall:</strong> 3 out of 5. If it weren't for the environmental damage of
this litter, I'd rate it at least a 4, but it's created through strip
mining. In addition, I'm not thrilled with the dust that it
generates. Other than that, this is my favorite litter to use and
until I find something better, is the litter I will have on standby
due to its odor control and siftability.</p>
<h1>PetCo Refillable Unscented Cat Litter</h1>
<p>Once I tried the Fresh Step, I thought about cheaper alternatives. I
am not generally a fan of PetCo due to their <a href="http://slumz.boxden.com/f610/petco-throws-live-baby-iguana-dumspter-2000314/">live animal policies policies</a>
as well as some horror stories I've heard about their store in DC, but
there's a store across the street from my apartment and they offer a
refill program where you can buy a box of cat litter, then refill it
for less than the price of a new box. I decided to give the </p>
<p><strong>Pros:</strong> This product is very inexpensive. It's the cheapest of all
the products I've used, and the cats don't seem to mind it. It looks
and feels very much like the Fresh Step, though less dusty.</p>
<p><strong>Cons:</strong> It doesn't do as good a job at odor control as the Fresh
Step. After about 3 days, despite daily cleanings, it smelled. It's
not as bad as some of the other litters, but it's a significant
difference from the Fresh Step. Additionally, it's another clay
litter, with no indication that it's not made from the strip mining
process.</p>
<p><strong>Overall:</strong> I'll give this product a 2 out of 5. Like the Fresh Step,
it loses a point for being environmentally unfriendly, and it looses
an additional point for being bad at odor control, but it's also
less dusty and cheaper than Fresh Step. I've had to knock another
point down for finding out that this product is just regular bags of
cat litter <a href="http://myplasticfreelife.com/forum/rant-rant-rant/not-bulk-litter-shame-on-petco/">dumped into a bin</a>.</p>
<h1>Conclusion</h1>
<p>I still have another half a container of the PetCo cat litter, which I
will use up in the next week. I also want to try a silica based cat
litter, one of the "Natural" labeled Fresh Step products, the
"Advanced Formula" World's Best Cat Litter, and the Blue Buffalo Cat
litter, so stay tuned for an update in a couple of months.</p>
<p>If none of these products works out, I will use the Fresh Step product
since it's the best at odor control. In the meantime, if you're a cat
litter company who wants me to try their product, don't hesitate to
contact me!</p>How Sirius/XM mis-handles customers2014-01-11T07:34:52-05:002014-01-11T07:34:52-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-11:/blog/2014/01/11/how-sirius-slash-xm-mis-handles-customers/<p>I had a recent experience with Sirius/XM that I think is a great
lesson in how not to handle your customers.</p>
<!--more-->
<p>Let's face it, Sirius/XM has an uphill battle in terms of
entertainment. They have dozens of music channels, but a subscription
to Pandora or Spotify will take …</p><p>I had a recent experience with Sirius/XM that I think is a great
lesson in how not to handle your customers.</p>
<!--more-->
<p>Let's face it, Sirius/XM has an uphill battle in terms of
entertainment. They have dozens of music channels, but a subscription
to Pandora or Spotify will take care of all your music needs. What's
left, and what's unique about the service is the premium content, the
things you can't hear anywhere else.</p>
<p><strong>+1 Sirius</strong>: Offer me a product I want</p>
<p>And that's why I enjoyed the service, for stuff I couldn't hear
anywhere else. But the service was a pain in the butt to use. I didn't
own a radio, so I had the Internet only service. But unlike a radio
stream from <a href="http://somafm.com/">SomeFM</a> or
<a href="http://www.live365.com">Live365</a>, or even my local <a href="http://www.wbai.org/">Pacifica station</a>, I couldn't just tune in and listen. I
had to use a complicated web app on my computer, and I couldn't keep
it on in the background because it would sign me out automatically
after a while.</p>
<p>I could use an app on my phone, but I couldn't listen to Sirius on my
Roku player, my Samsung Smart TV, my Boxee box or my homebuilt XBMC
box. Somehow, three separate video services are able to deliver
thousands of TV shows and movies to me, but Sirius can't deliver audio
to me. Roku has sold over <a href="http://techcrunch.com/2013/04/10/roku-hits-5m-streaming-players-sold-in-the-u-s-has-streamed-8b-videos-and-music-tracks/">five million</a>
units, but Sirius still seems focused on the car market, rather than
offering entertainment anywhere.</p>
<p><strong>-1 Sirius:</strong> Offer your product as many places as possible</p>
<p>I mentioned earlier that Sirius has music, but it also has talk
services from NPR, PRI and the BBC. But like music, all of those
services offer live streaming, or better yet, podcasts. With podcasts,
I can listen to the shows whenever I want, stop them, rewind them, or
even share them with friend- all for free.</p>
<p>With the Sirius/XM premium content, I can't even choose to stream the
content when I want. By comparison to modern content delivery systems,
Sirius feels like a step backwards.</p>
<p><strong>-1 Sirius</strong>: Offer the product in a form your customers want it</p>
<p>Because Sirius is such a pain to use, I simply forgot to turn it on
for about six months. Every month, I dutifully (auto)paid my
bill. But after my credit card changed, the autobill stopped working,
and I received a call at 6pm someone saying they were from Sirius/XM
and wanting to talk to me about my bill.</p>
<p>I told her that I didn't have time to talk at the moment, but that I
didn't want to use the service anymore, so I'd like to cancel. She
said that she'd done it, but I still had the matter of my outstanding
bill. I told her I'd take care of that online.</p>
<p>She ended the conversation with "I'll call you tomorrow." I told her I
didn't want her to call tomorrow- that I'd take care of it on my own.</p>
<p>The next day, also while preparing dinner, I received a call from a
different gentleman claiming to represent Sirius/XM, and wanting to
talk to me about my bill. This time, I'd had enough, and just hung up
on him.</p>
<p><strong>-1 Sirius</strong>: Don't harass your customers!</p>
<p>The next morning I went online to straighten everything out. Online,
it showed my service was still active, and that I indeed owed money
for my week and a half of service.</p>
<p><strong>-1 Sirius</strong>: Customer service rep lied about canceling my service!</p>
<p>On the website, it says you cannot cancel online, but need to call
between certain hours.</p>
<p><strong>-1 Sirius</strong>: Making the customer jump through hoops to cancel</p>
<p>I called customer service later in the day and spoke with a very
pleasant young woman named Sarah, who helped me through the
cancellation process. She asked me why I wanted to cancel, and I told
her about the customer service reps. She was very understanding, and
even took off the pro-rated $4.95 that I owed for my partial month of
service (that was only a third of what the other customer reps had
said I owed, btw), in hopes that I might join back up with Sirius/XM
in the future.</p>
<p>At the end of the conversation, I honestly considered it. After all,
there was some good programming on Sirius, but I needed a few days to
think about it.</p>
<p><strong>-1 Sirius</strong>: Lying about how much the customer owes</p>
<p><strong>+1 Sirius</strong>: Good customer service when you call to cancel</p>
<p><strong>+1 Sirius</strong>: Taking off the $5 from my bill</p>
<p>I felt good about my Sirius XM experience until just a few hours later
when yet another hard-to-hear call from yet another foreign person
about my service. This time, after going back and forth about not
being able to hear the person on the other end of the line, I finally
understood that she was calling me to inform me that my service was
canceled!</p>
<p>Absolute madness.</p>
<p><strong>-1 Sirius</strong>: Harassing your customers after they leave</p>
<p>That brings our tally to -4 against Sirius/XM, and a great lesson in
how not to treat your customers.</p>You don't need to worry about GPS2014-01-07T11:03:21-05:002014-01-07T11:03:21-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-07:/blog/2014/01/07/you-dont-need-to-worry-about-gps/<p><a href="http://en.wikipedia.org/wiki/Gps">GPS</a> is virtually synonymous
with navigation, but GPS exists at the whim of the US government, so
shouldn't I be concerned? The answer is that I am somewhat concerned,
but that I also do my part in trying to reduce our dependence on that
system.</p>
<!--more-->
<p>I'll venture to bet that …</p><p><a href="http://en.wikipedia.org/wiki/Gps">GPS</a> is virtually synonymous
with navigation, but GPS exists at the whim of the US government, so
shouldn't I be concerned? The answer is that I am somewhat concerned,
but that I also do my part in trying to reduce our dependence on that
system.</p>
<!--more-->
<p>I'll venture to bet that everyone reading this knows what GPS is, but
I suspect fewer people realize that GPS is entirely owned and
controlled by the US Department of Defense. In other words, the
millions of GPS enabled devices work because the US government allows
them to work, and if the US decided to shut the system down, even
temporarily, it could mean disaster. Am I worried? Not really.</p>
<p>** Disabling GPS Would be a Financial Disaster **</p>
<p>The GPS system is used worldwide, in everything from airplanes to cell
phones. If that system was degraded sufficiently, or shut down even
temporarily, it would cause a global financial disaster.</p>
<p>Even if the US were to only select certain satellites and to degrade
the signal over certain regions, there would likely be unforeseen
consequences which would eventually effect the US financially.</p>
<p>In addition to any immediate consequences, there would also be long
term issues raised around US reliance, which is a topic of increasing
concern internationally, especially after recent NSA revelations.</p>
<p>** GPS is Not the Only Game in Town**</p>
<p>Even if the US were to shut down its system, the US's GPS system is
not the only satellite navigation system. Russia's GLONASS system,
China's Beidou system, and Europe's upcoming Galileo system are all
offering services which are similar in functionality and quality to
GPS, and with <a href="http://www.engadget.com/2008/05/28/nemerix-scores-patent-for-single-chip-gps-glonass-galileo-rece/">all in one</a>
chips coming out, there will be even less incentive to degrade any one
system's service.</p>
<p>** Ground Based Location Systems Exist **</p>
<p>In addition to satellite navigation systems, we can, and often do use
ground based navigation systems. Airplanes use a variety of land-based
augmentation systems such as
<a href="http://en.wikipedia.org/wiki/Wide_Area_Augmentation_System">WAAS</a> and
cell phones can determine their location by use of <a href="http://wrongfulconvictionsblog.org/2012/06/01/cell-tower-triangulation-how-it-works/">cell tower triangulation</a>.</p>
<p>In addition to these methods, by looking at other signals, such as
<a href="http://www.zdnet.com/blog/networking/how-google-and-everyone-else-gets-wi-fi-location-data/1664">Wifi access points</a>,
one can lookup their location. Both Google and Apple use a
combination of cell tower data and wifi access points in augmenting
their GPS systems.</p>
<p>** The Mozilla Location Service **</p>
<p>There have been a number of cell and wifi collection services. All of
them have so far been either technically lacking or proprietary in
nature. That's hopefully changing with the <a href="https://wiki.mozilla.org/CloudServices/Location">Mozilla Location
Service</a>, a new
Mozilla project to provide an alternative to the popular proprietary
services, including source code to both the server and client and,
<a href="https://wiki.mozilla.org/CloudServices/Location/Privacy">once the privacy issues are sorted
out</a>, a
downloadable database of locations.</p>
<p>My personal contribution to the project is that I run their data
collection client,
<a href="https://github.com/mozilla/MozStumbler/releases">MozStumber</a> on my
Android phone, and I encourage others to do the same.</p>
<p>** A Free GPS Service?**</p>
<p>Would it be better if we had our own navigation system owned by the
community, used as a common resource in the same way that Free/Open
Source software, and the OpenStreetMap project serve the community. Of
course it would, but do to that, we'd need millions of dollars in
development, construction, launch and maintenance of the satellites
needed for the effort. Even with advanced in amateur rocketry and
<a href="http://www.kurzweilai.net/how-to-launch-your-own-homemade-satellite">satellites</a>,
I don't think that this project will be feasible in the near future,
but it's also not necessary with so many existing alternatives.</p>
<p>So don't worry about knowing where you are, you're covered.</p>The World Cares About OpenStreetMap2014-01-05T23:26:45-05:002014-01-05T23:26:45-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-05:/blog/2014/01/05/the-world-cares-about-openstreetmap/<p>[<strong>UPDATE</strong>] I will stop updating this, but after four translations,
syndication by The Guardian, and a video interview on Slashdot... I'm
speechless.</p>
<p>Wow, what a response! My <a href="http://blog.emacsen.net/blog/2014/01/04/why-the-world-needs-openstreetmap/">last post</a>, received over 120,000 unique
visitors, over <a href="https://twitter.com/intent/tweet?original_referer=http%3A%2F%2Fblog.emacsen.net%2Fblog%2F2014%2F01%2F04%2Fwhy-the-world-needs-openstreetmap%2F&text=Why%20the%20world%20needs%20OpenStreetMap%20-%20emacsen.net&tw_p=tweetbutton&url=http%3A%2F%2Fblog.emacsen.net%2Fblog%2F2014%2F01%2F04%2Fwhy-the-world-needs-openstreetmap%2F">500 tweets</a>,
was #1 on <a href="https://news.ycombinator.com/item?id=7015294">Hacker News</a>,
was #2 on
<a href="http://www.reddit.com/r/technology/comments/1ugi39/why_the_world_needs_openstreetmap/">/r/technology</a>
on Reddit …</p><p>[<strong>UPDATE</strong>] I will stop updating this, but after four translations,
syndication by The Guardian, and a video interview on Slashdot... I'm
speechless.</p>
<p>Wow, what a response! My <a href="http://blog.emacsen.net/blog/2014/01/04/why-the-world-needs-openstreetmap/">last post</a>, received over 120,000 unique
visitors, over <a href="https://twitter.com/intent/tweet?original_referer=http%3A%2F%2Fblog.emacsen.net%2Fblog%2F2014%2F01%2F04%2Fwhy-the-world-needs-openstreetmap%2F&text=Why%20the%20world%20needs%20OpenStreetMap%20-%20emacsen.net&tw_p=tweetbutton&url=http%3A%2F%2Fblog.emacsen.net%2Fblog%2F2014%2F01%2F04%2Fwhy-the-world-needs-openstreetmap%2F">500 tweets</a>,
was #1 on <a href="https://news.ycombinator.com/item?id=7015294">Hacker News</a>,
was #2 on
<a href="http://www.reddit.com/r/technology/comments/1ugi39/why_the_world_needs_openstreetmap/">/r/technology</a>
on Reddit, and a <a href="https://duck.co/forum/thread/4988/link-why-the-world-needs-openstreetmap">mention by the CEO of
DuckDuckGo</a>,
and more.</p>
<!--more-->
<p>We also saw the number of new user accounts per day triple from ~33
accounts per day to just over 120 accounts per day, which doesn't
include the buildup and the cool-down, which also exceeded our signup
rate.</p>
<p>I also received several nice pieces of email from folks who read the
post and either had followup questions, or just wanted to thank me for
writing it.</p>
<p>I'm writing this post to express my gratitude to everyone.</p>
<p>With any group, it's easy to lose perspective, or get lost in an echo
chamber of your peers. I wasn't sure who would care to read about a
project they likely never used, but based on the response, I think the
answer is that there are a lot of people. That's really validating and
helps me get through the tougher times in the project.</p>
<p>For new folks, I won't try to re-hash the many new user introduction
guides out on the web, but in future posts, I'll try to talk more
about the internals of the project- how things work behind the
scenes. For regular editors, I'll be talking about some exciting work
that I'm doing to make our project better.</p>
<p>Stay tuned because it's going to be a really fun ride.</p>
<p>Thanks again for the support!</p>
<p><strong>Update</strong> 144k unique views, >700 tweets, >100 Google+ posts, and nearly 350 Facebook posts.</p>
<p>Oh, and <strong><a href="http://gizmodo.com/why-the-world-needs-openstreetmap-1495412839">GIZMODO SYNDICATED IT!</a></strong></p>Why the world needs OpenStreetMap2014-01-04T20:46:25-05:002014-01-04T20:46:25-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-04:/blog/2014/01/04/why-the-world-needs-openstreetmap/<p>Every time I tell someone about OpenStreetMap, they inevitably ask
"Why not use Google Maps?". From a practical standpoint, it's a
reasonable question, but ultimately this is not just a matter of
practicality, but of what kind of society we want to live in. I
discussed this topic in a …</p><p>Every time I tell someone about OpenStreetMap, they inevitably ask
"Why not use Google Maps?". From a practical standpoint, it's a
reasonable question, but ultimately this is not just a matter of
practicality, but of what kind of society we want to live in. I
discussed this topic in a 2008 talk on OpenStreetMap I gave at the
first MappingDC meeting. Here are many of same concepts, but expanded.</p>
<!--more-->
<p>In the 1800s, people were struggling with time, not how much of it
they had, but what time it was. Clocks existed, but every town had its
own time, "Local time", which was synchronized by town clocks, or more
often than not, church bells. Railway time, then eventually Greenwich
Mean Time eventually supplanted all local time, and most people today
don't think about time as anything but universal. This was
accomplished in the US by adoption first of the railroads, and then by
universities and large businesses.</p>
<p>The modern day time dilemma is geography, and everyone is looking to
be the definitive source. Google spends <a href="http://www.asymco.com/2012/12/18/how-much-to-maps-cost-and-what-are-they-worth/">$1 billion annually</a>
maintaining their maps, and that does not include the 1.5 billion
Google spent <a href="http://techcrunch.com/2013/06/11/its-official-google-buys-waze-giving-a-social-data-boost-to-its-location-and-mapping-business/">buying Waze</a>. Google
is far from the only company trying to own everywhere, as Nokia
purchased Navtek and TomTom and
Tele Atlas try to merge. All of these companies want to become the
definitive source of what's on the ground.</p>
<p>That's because what's on the ground has become big business. With
GPSes in every car, and a smartphone in every pocket, the market for
telling you where you are and where to go has become fierce.</p>
<p>With all these companies, why do we need a project like OpenStreetMap?
The answer is simply that as a society, no one company should have a
monopoly on place, just as no one company had a monopoly on time in
the 1800s. Place is a shared resource, and when you give all that
power to a single entity, you are giving them the power not only to
tell you about your location, but to shape it. In summary, there are
three concerns- who decides what gets shown on the map, who decides
where you are and where you should go, and personal privacy.</p>
<p>Who decides what gets displayed on a Google Map? The answer is, of
course, that Google does. I heard this concern in a meeting with a
local government in 2009- they were concerned about using Google Maps
on their website because Google makes choices about which businesses
to display. They were right to be concerned about this issue since a
government needs to remain impartial and by outsourcing their maps,
they hand the control over to a third party.</p>
<p>It seems inevitable that Google will monetize geographic searches,
with either premium results, or priority ordering, if they haven't
done so already (ie is it a coincidence than when I search for
"breakfast" near my home, the first result is "SUBWAY® Restaurants").</p>
<p>Of course Google is not the only map provider, they're just one
example. The point is that when you use any map provider, you are
handing them the controls- letting them determine what features get
emphasized, or what features may not be displayed at all.</p>
<p>The second concern is about location. Who defines where a neighborhood
is, or whether or not you should go. This issue was brought up by the
<a href="https://www.aclu.org/blog/racial-justice-criminal-law-reform-technology-and-liberty/your-turn-turn-navigation-application">ACLU</a>,
where a map provider was providing routing (driving/biking/walking
instructions) and used what they determined to be safe or dangerous
neighborhoods as part of their algorithm. This begs the question of
who determines what makes a neighborhood safe or not, or whether safe
is merely a codeword for something more sinister.</p>
<p>Right now, Flickr collects neighborhood information based on
photographs which they expose <a href="http://code.flickr.net/2008/10/30/the-shape-of-alpha/">through an API</a>. They use this information to suggest tags for your photograph, but it would be
possible to use neighborhood boundaries in a more subtle way in order
to effect anything from traffic patterns to real estate prices,
because when a map provider becomes large enough, they become
the source of "truth".</p>
<p>Lastly, these map providers have an incentive to collect information
about you in ways that you may not agree with. Both Google and Apple
collect your location information when you use their services. They can
use this information to improve their map accuracy, but Google has
already announced that is going to use this information to <a href="http://digiday.com/platforms/google-tracking/">track the correlation between searches and where you go</a>. With <a href="http://news.cnet.com/8301-1035_3-57510994-94/google-500-million-android-devices-activated/">500 million Android phones</a>, this is an enormous amount of information collected on the individual level about people's habits whether they're taking a casual stroll, commuting to work, going to their doctor, or maybe attending a protest. Certainly we can't ignore the societal implication of so much data in the hands of a single entity, no matter how <a href="http://en.wikipedia.org/wiki/Don%27t_be_evil">benevolent they claim to be</a>. Companies like Foursquare use <a href="http://engineering.foursquare.com/2014/01/03/the-mathematics-of-gamification/">gamification</a> to overlay what is essentially a large scale data collection process, and even Google has gotten into the game of gamification with <a href="https://en.wikipedia.org/wiki/Ingress_%28game%29">Ingress</a>, a game which overlays an artificial world onto this one and encourages users to collect routing data and photo mapping as part of effort to either fight off, or encourage, an alien invasion.</p>
<p>Now that we have identified the problems, we can examine how
OpenStreetMap solves each of them.</p>
<p>In terms of map content, OpenStreetMap is both neutral and transparent.
OpenStreetMap is a wiki-like map that anyone in the world can edit. If a
store is missing from the map, it can be added in, by a store owner or
even a customer. In terms of display (rendering), each person or company
who creates a map is free to render it how they like, but the main map
on <a href="http://www.openstreetmap.org">OpenStreetMap.org</a> uses FLOSS
(Free/Libre Open Source Software) rendering software and a liberally
licensed stylesheet which anyone can build on. In other words, someone
who cares can always create their own maps based on the same data.</p>
<p>Similarly, while the most popular routers for OpenStreetMap are FLOSS,
even if a company chooses another software stack, a user is always free
to use their own routing software, and it would be easy to compare
routing results based on the same data to find anomalies.</p>
<p>And lastly, with OpenStreetMap data, a user is free to download some, or
all of the map offline. This means that it's possible to use
OpenStreetMap data to navigate without giving your location away to
anyone at all.</p>
<p>OpenStreetMap respects communities and respects people. If you're not
already contributing to OSM, consider helping out. If you're already a
contributor- Thank You.</p>
<p><strong>Update</strong> Wow, what a <a href="http://blog.emacsen.net/blog/2014/01/05/the-world-cares-about-openstreetmap/">response!</a></p>Learning Math Earlier With Computers2014-01-04T00:15:24-05:002014-01-04T00:15:24-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-04:/blog/2014/01/04/learning-math-earlier-with-computers/<p>Writing the previous post got me thinking about the nature of
how programmers use programs and how it relates to teaching in
general.</p>
<!--more-->
<p>A couple of years ago, I wrote a little prototype address book that
accounted for my misspellings. The idea was that if someone had a
name like …</p><p>Writing the previous post got me thinking about the nature of
how programmers use programs and how it relates to teaching in
general.</p>
<!--more-->
<p>A couple of years ago, I wrote a little prototype address book that
accounted for my misspellings. The idea was that if someone had a
name like Karalyne, that even if I spelled it "Caroline", it would
find it.</p>
<p>To do so, I used the Soundex algorithm in Python:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="kn">import</span> <span class="nn">fuzzy</span>
<span class="o">>>></span> <span class="n">soundex</span> <span class="o">=</span> <span class="n">fuzzy</span><span class="o">.</span><span class="n">Soundex</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">soundex</span><span class="p">(</span><span class="s1">'Caroline'</span><span class="p">)</span>
<span class="s1">'C6450'</span>
<span class="o">>>></span> <span class="n">soundex</span><span class="p">(</span><span class="s1">'Karalyn'</span><span class="p">)</span>
<span class="s1">'K6450'</span>
</code></pre></div>
<p>Then I used the Levenshtein algorithm to determine the best match, and
as the user typed, the matches would become better.</p>
<p>When I wrote the program, and even today, I have no understanding of
exactly how Soundex works. I know that it transforms the string into
a set of values which correspond to how it sounds, but I don't need
to understand its implementation to use it.</p>
<p>That's the way most libraries work- one doesn't need to understand
them in order to use them.</p>
<p>In <a href="http://www.ted.com/talks/conrad_wolfram_teaching_kids_real_math_with_computers.html">this TED talk</a>,
Conrad Wolfram argues that this is the same way that we should be
teaching math to children, with an emphasis on usage first, and
understanding afterwards.</p>
<p>His examples include geometry, but the same arguments could be made
about statistical analysis.</p>
<p>Most people working with a dataset will need to know what questions
to ask- determining the type of data and knowing what questions to
ask is the most important thing. Figuring out whether or not the data
fits a certain model will help you determine the type of analysis
that needs to be done next.</p>
<p>If we taught statistics this way, with an emphasis on the analysis,
rather than the process, we could teach statistics not in college, as
<a href="http://www.ted.com/talks/arthur_benjamin_s_formula_for_changing_math_education.html">Arthur Benjamin</a>
suggests, but as earlier, even as early as Algebra. The concepts in a
subject like statistics are concrete, they're tangible, and it would
be possible for students to be as comfortable using them even before
they understand them, just as I'm comfortable using Soundex, even when
I have only the most rudimentary understanding of how it works.</p>Python for Adults2014-01-03T11:42:00-05:002014-01-03T11:42:00-05:00Serge Wroclawskitag:blog.emacsen.net,2014-01-03:/blog/2014/01/03/python-for-adults/<p>My girlfriend has always wanted to program. She even took C++ in high
school and college, but never quite "Got it", leaving her with a bad
taste for computers and programming. Still, when I lent her my copy of
<a href="http://www.ccs.neu.edu/home/matthias/BTLS/">The Little Schemer</a>, she
took to it pretty easily, so I …</p><p>My girlfriend has always wanted to program. She even took C++ in high
school and college, but never quite "Got it", leaving her with a bad
taste for computers and programming. Still, when I lent her my copy of
<a href="http://www.ccs.neu.edu/home/matthias/BTLS/">The Little Schemer</a>, she
took to it pretty easily, so I knew she had the potential to be able to
program, despite her past difficulties.</p>
<!--more-->
<p>So then the question becomes what language to teach her. While Scheme
would be a nice choice in theory, despite more than three decades as a
teaching language, it's still not popular in the mainstream. While this
wouldn't be important in a classroom setting, adults have a different
set of needs for programming. Notably, they need to be able to apply
their skills quickly to something in their life.</p>
<p>That's why Python fits the bill so well. First off, its simple. It's
syntax is absolutely minimalist, as are its features. I call Python by
"80% Language", in that 80% of all tasks I need to do, I can do with
Python. It doesn't have <a href="http://www.robertsosinski.com/2008/12/21/understanding-ruby-blocks-procs-and-lambdas/">Ruby Blocks</a>,
or <a href="http://www.python.org/dev/peps/pep-0245/">Interfaces</a>, or <a href="http://stackoverflow.com/questions/267862/what-makes-lisp-macros-so-special">Lisp Macros</a>, Python is arguably the best general purpose language out there.</p>
<p>The key to Python's success is three part. The first of those is design.
Python's roots as a teaching or prototyping language meant that there is
an emphasis on simplicity and elegance. It's very light on syntax, it
supports Object Oriented programming, but does not demand it, and
operations that are performed often, like array indexing, hash table
lookups and mapping have special syntax to make them more convenient.</p>
<p>Python also stays firmly in the general purpose language camp.
<a href="http://www.r-project.org/">R</a> has become increasingly popular for
statistics work in the last five years, and while R does have
<a href="http://cran.r-project.org/doc/manuals/R-exts.html">extensions</a>, it is
designed for data analysis. Perl started off its life as a text
processing and scripting language, and while it's grown out of that
role, it still has those concepts embeddeded within it. Python changes
slowly over time and borrows from other languages while staying true to
itself. For example, Python recently adopted <a href="http://www.python.org/dev/peps/pep-3156/">Asyncronous
IO</a> in the core, but does not
require that any program use it.</p>
<p>The second strength Python has are its libraries. While Python has fewer
libraries than Perl, it offers more simple, practical interfaces to more
systems. For example, I mentioned R earlier, and Python programmers can
interface directly with R through <a href="http://rpy.sourceforge.net/">RPy</a>,
while still being able to use simple web accessing frameworks like
<a href="http://docs.python-requests.org/en/latest/">Requests</a>, and powerful
phonetic algorithms such as those found in
<a href="https://pypi.python.org/pypi/Fuzzy">Fuzzy</a>. In fact, whole Python
distributions have been created around nothing but packaging up the best
libraries for a particular task, such as <a href="http://www.scipy.org/">SciPy</a>,
and <a href="http://biopython.org">Biopython</a>.</p>
<p>The third, and perhaps most important strength of Python is its user
community. I'm sure this will be the most controversial part of this
post, but I've found the Python community has bar-none the most
supportive users. This is not by accident, but part of Python's legacy
and current commitment to inclusion. Python came from a teaching
language background, and documentation was, and continues to be part of
that legacy. Python is used as a teaching language in <a href="http://www.python.org/workshops/2000-01/proceedings/papers/elkner/pyYHS.html">High Schools</a>
as well as
<a href="http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-189-a-gentle-introduction-to-programming-using-python-january-iap-2011/">MIT</a>.</p>
<p>In addition, the Python Foundation focuses a lot of attention and energy
into community diversity through its <a href="http://www.python.org/community/diversity/">Diversity Statement</a>, as well as
commitment to bring women into the community, both through their own
local communities (<a href="http://www.pyladies.com/">PyLadies</a>) but also
focusing on bringing that diversity into mainline events such as PyCon.
The net effect is that Python is not only welcoming to women, but has a
general welcoming atmosphere to people of virtually any background.</p>
<p>If someone is considering Python, especially if they have a scientific
background, I suggest watching <a href="http://www.youtube.com/watch?v=F4rFuIb1Ie4">Science and Python</a>, and jumping in with
<a href="http://learnpythonthehardway.org/">Learn Python the Hard Way</a> followed
with <a href="http://newcoder.io/">New Coder</a>, which is described in depth at
<a href="http://pyvideo.org/video/1782/sink-or-swim-5-life-jackets-to-throw-to-new-code">this talk</a>
by Lynn Root.</p>Reintroductions2012-06-09T08:30:00-04:002012-06-09T08:30:00-04:00Serge Wroclawskitag:blog.emacsen.net,2012-06-09:/blog/2012/06/09/reintroductions/<p>Wow, I must have really touched on something in my last post
(<a href="http://blog.emacsen.net/blog/2012/06/07/observations-from-a-python-workshop/">Observations at a Python workshop</a>), it's received over
20k unique visitors in the last twenty four hours, which is a lot from
a blog which until yesterday, didn't exist.</p>
<p>For those people who don't know me, I thought …</p><p>Wow, I must have really touched on something in my last post
(<a href="http://blog.emacsen.net/blog/2012/06/07/observations-from-a-python-workshop/">Observations at a Python workshop</a>), it's received over
20k unique visitors in the last twenty four hours, which is a lot from
a blog which until yesterday, didn't exist.</p>
<p>For those people who don't know me, I thought I'd give some
introductions.</p>
<p>My name is Serge Wroclawski, I'm a long standing member of the Free
Software community. I spent 12 years of my life as a Unix system
administrator, mostly working for academic government institutions
such as NIH and NASA.</p>
<p>I've quit my job to work on my passion, which is to help the various
communities involved in software, culture, education- basically to
make works available to anyone and everyone who wants it, without
restrictions.</p>
<p>To that end, I'm in the process of founding a new non-profit dedicated
to Free Culture and making it easier for artists to add their work
into Free Culture. If anyone would be interested in helping in this
effort, please feel free to <a href="http://blog.emacsen.net/contact">contact me</a>.</p>
<p>I'm also moving to New York City later this summer and would love to
meet some new people there, especially those in either the arts or
technical community, so if you're in/around NYC drop me a line!</p>
<p>Thanks again everyone for reading/subscribing. I hope to have lots of
posts in the days to come; my personal goal is one not less than every
two weeks.</p>
<p>And if anyone is curious, my <a href="http://emacsen.livejournal.com">old blog</a> is still around for now,
though I am not updating it, and it's full of comment spam.</p>Observations from a Python Workshop2012-06-07T23:09:00-04:002012-06-07T23:09:00-04:00Serge Wroclawskitag:blog.emacsen.net,2012-06-07:/blog/2012/06/07/observations-from-a-python-workshop/<p>I heard about a Python tutorial that was going on this weekend, and
with well over a decade of Python experience, I decided to volunteer
to help students. I hadn't realized it was a women's workshop put on
by <a href="http://pyladies.com">PyLadies</a>, but I'm glad I went because I came out of …</p><p>I heard about a Python tutorial that was going on this weekend, and
with well over a decade of Python experience, I decided to volunteer
to help students. I hadn't realized it was a women's workshop put on
by <a href="http://pyladies.com">PyLadies</a>, but I'm glad I went because I came out of
the event with a number of observations that extend out to the Free
Software community as a whole.</p>
<p>It's no secret that our community isn't great at encouraging women
contributors. <a href="http://www.networkworld.com/community/node/58218">Various surveys</a> show that between 1.5
and 5% of Free Software contributors are women. That is low even
compared to the 12-25% of <a href="http://www.quora.com/What-is-the-percentage-of-women-in-computer-science-generally">women in computer science</a>. And
while we all cringe at incidents like the Golden Gate Ruby Conference
<a href="http://www.ultrasaurus.com/sarahblog/2009/04/gender-and-sex-at-gogaruco/">fiasco</a>, and the <a href="http://thenextweb.com/us/2012/03/20/sqoot-loses-sponsors-following-misogynistic-description-of-their-api-jam-event/">Squoot Jam</a>, these
incidents alone don't explain the lack of participation by women in
our community.</p>
<h2>The Audience Is Out There</h2>
<p>The class was well attended. Seventeen women and one man came to learn
how to program in Python. Considering that Python materials are
readily available in books, on the web and in videos, this heavy
attendance shows that there are people who want to learn and that they
will if they're presented with the opportunity.</p>
<h2>Motivation</h2>
<p>In one of the classic programming texts (I forget which one), the
author says that very few people who learn to program should be
programmers, but that he'd like to see more programming done by those
of other professions. The workshop would have made him proud, as every
student I talked to shared the sentiment with me that they hoped to
learn Python in order to do their job more effectively, by learning to
collect, sort or process data in new and more efficient ways.</p>
<h2>The Students Care</h2>
<p>During the first half of the class, the students were engaged, and
excited. They asked questions, and worked with the instructor, and
with each other, to help fill in the gaps. Sometimes they'd ask a
volunteer for help, but largely the questions I received were
grammatical rather than conceptual. </p>
<h2>The Unsure Student</h2>
<p>By and large the students seemed comfortable with the material and
engaged with the instructor, but there were exceptions.</p>
<p>One of the students who sat next to me was a young woman who was far
ahead of the class. She was easily an hour ahead of the other students
in the progress she'd made with the final project when she asked me
for help.</p>
<p>Looking at her screen, she'd made significant progress towards
finishing the project, but when she asked me a question she giggled
and told me she had no idea what the project was about, or what to do.</p>
<p>Why?</p>
<p>I still don't know the reason. <a href="http://www.psychologytoday.com/blog/the-science-success/201101/the-trouble-bright-girls">Studies</a> have shown that
especially bright women have this reaction to difficult material and
that women learning computer science do better in an
<a href="http://www.singlesexschools.org/advantages-equity.htm">single sex</a> classroom (though those studies have
<a href="http://www.sciencemag.org/content/333/6050/1706.full">critics</a>). But I can say that I personally found this
experience surprising, and I wondered afterwards why this extremely
competent woman in her 20s would react this way after clearly
mastering the material.</p>
<h2>The Presence of a "Helpful" Jerk</h2>
<p>I mentioned earlier that the first half of the class had a lot of
student engagement, with questions and joking.</p>
<p>After lunch, a new volunteer joined the classroom, someone I'll call
Dave.</p>
<p>Dave was really excited by the class, and, like me and the other
volunteers, had a lot of experience with Python.</p>
<p>Dave expressed his enthusiasm for the material by asking a lot of very
technical questions to the instructor. Largely, these questions seemed
designed to show off his own knowledge, or to challenge the
instructor. In other words, Dave was the <a href="http://kev.inburke.com/kevin/how-not-to-ask-questions-at-conference">jerk at the conference</a>.</p>
<p>And Dave's behavior had a noticable effect on the workshop. Students
tuned out, became distracted, asked less questions, and had a lot more
difficulty following the material.</p>
<p>While the students tuned out, I felt myself getting angry. I saw
Dave's behavior as a challenge, a call to action. And it was at that
moment that the gender difference became most apparent to me.</p>
<p>To me, and I suspect most men, we'd see Dave's behavior as a call to
arms. Dave was raising the stakes and even though his behavior was
clearly rude, it also demonstrated that he had a mastery over the
material.</p>
<p>But to this classroom, Dave's behavior not only ground the timing of
the class down to a halt, it also seemed to have an effect on the
students, who had until then been very involved in the workshop.</p>
<h2>Lessons Learned</h2>
<p>I'd never really believed in single sex education, and I'd really
never believed that the gender difference in computer science
education was due to lack of interest. Just as I've never heard an
outcry about how few <a href="http://www.menteach.org/node/34">male teachers</a> exist, I never
understood the outcry about the lack of women programmers or
sys-admins.</p>
<p>But my experience last weekend changed my view. One of the students
from the workshop came to the <a href="http://meetup.dcpython.org">DC Python</a> meetup, and I saw
that women only groups are not creating a more insular community of
women, but rather offering these women a more comfortable entrance
into our general community, and so I say more <a href="http://linuxchix.org">LinuxChix</a>,
more <a href="http://pyladies.com">PyLadies</a>, and more <a href="http://ladieslearningcode.com">Ladies Learning Code</a>.</p>
<p>The jerk factor also reminded me of why we in the Free Software
community need to be taking issues of civility seriously, and not
letting the idea of "free speech" get in the way of maintaining a safe
place for discussion. I've seen groups fail in both ways, by allowing
jerks to run the roost, and with draconian and arbitrary moderation. I
can't recommend <a href="http://www.artofcommunityonline.org">Art of Community</a> enough.</p>