Thursday, February 27, 2025

Appletalk Routing in 2025

As I've continued to play around with retro computing environments, based largely in virtual labs built with various incarnations of Qemu, I've run into some situations where I either needed or wanted to route some native Appletalk servers over IP networks, where Appletalk routers don't exist in between - or, at the very least, the routers don't have Appletalk services enabled, and I am not in control of the routers. It's also possible that I'm trying to contain my ancient computing environments enough that I both don't expose my overall environment to needless risk, and that I keep an InfoSec team happy that my activities are not doing that. So, how is one to overcome the obstacles of Layer 3 segmentation and Appletalk routing in 2025?

Simple: Linux. Okay, of course Linux - that's obvious, let's get more specific.

The biggest item you'll need - running on Linux, of course - is Netatalk. Netatalk is an open source Apple compatibility software suite for Linux, specializing in supporting some of the older Mac technologies, most notably the Apple Filing Protocol (AFP). In fact, version 3.x and 4.x of Netatalk really only support AFP. However, if you go back to the 2.x release, which still has some level of active maintenance on it (git code was updated in October 2024), you'll find it supports quite a wide range of Apple services - beyond AFP, it also supports printing (PAP), time synchronization (timelord), and, the one we're most interested in, Appletalk (atalk). Appletalk was a network protocol similar to Internet Protocol (IP) and IPX/SPX, and included features like network and node addressing that allowed it to inter-connect Apple systems over a wide range of networks, distances, media types, etc.

Several vendors included Appletalk routing in their products, as well. Cisco routers had it built in for years - and may still have the feature present (I haven't been on a Cisco router in a decade or more, so I'm not sure about that). Also, notably, Microsoft Windows NT 4.0 Server had a service that could be installed called "Services for Macintosh," which included, among other things (more on those in a future post), the capability to set up Windows to create Appletalk Zones (the equivalent of IP subnets) and route between them. Ultimately IP won out over the likes of Appletalk (and IPX/SPX, and a few others), but not before many organizations built out rather large Appletalk networks connecting Macintosh as well as other Apple platforms together with this protocol.

Thursday, January 16, 2025

Reliving the Glory Days

I've recently been inspired by a couple of YouTube channels - The 8-bit Guy, The Serial Port and clabretro - to relive the glory days of my youth and the infancy of the Internet. I grew up in the 1980s and 1990s when the Internet was just getting going - when the folks creating web sites on the likes of GeoCities could create sites every bit as good as major corporations, when e-mail was cutting edge, and when AOL Instant Messenger was how kids spent their free time. Those were the glory days.

I have a lot of nostalgia about that time, and some memories tied to how I explored this new digital, online world, and, as I've watched those YouTube channels explore some of the really fun things like setting up a dial-up ISP, or configuring a Sun Sparc server, I had a strong desire to recreate some of the things that I used to do, as well.

And, of course, the question becomes: How can I make my retelling of my story unique? After all, those other YouTubers are already making those videos, so why should anyone pay attention to another guy doing much the same? Maybe no one will pay attention, and that's fine - the journey is primarily for me. But, in case other folks are interesting, I did come up with some ways that my recollecting of my experiences might be unique:

  • I intend to write it out as much as possible, and I don't even know if I'll make videos. Maybe I will at some point - that's never really been my thing, but, who knows.
  • I intend to write out the steps in great detail, so that others can reproduce it.
  • While many of the other channels are focused on procuring real hardware - things like Cisco 2600 series routers, line cards, terminal servers, Sun servers, etc. - I intend to recreate it with as much realism as possible in a virtual world. While that may sound like it isn't really reliving the past, I think it may be more accessible for others, and, hey, it's what I can do - I don't have the resources or attic/basement/closet space to procure a bunch of old hardware.
  • The other reason it's okay to do it virtually is that my nostalgia has more to do with how it looked to me, the end user, and not how it was implemented by a provider. While I certainly plan to dig into the back-end details of setting up proxy, mail, dial-up servers, etc., I didn't do any of that as a primary school kid in the 90s. I was just a user, and I'll be more focused on the items that help replicate what it was like as a user.

In addition to replicating and reliving my early Information Age experiences, I'd also like to explore some of the technologies that I was not given access to. I was a user of early Apple production (IIe, Mac Classic, Performa) and then early IBM PC-compatible platforms (DOS 6 + Windows 3.1, Windows 95/98, XP), and some Intel-based Linux distributions (Red Hat 6). My exploration of some of the retro YouTube channels has opened my eyes to some of the platforms that, for a combination of reasons, I completely missed - things like the Commodore platform, BASIC, OS/2. Many of these are available as downloads and can be run on Intel-compatible hardware, either directly or in readily-available emulators (Qemu).

Thursday, June 2, 2016

Application Delivery the Free and Open Source Way

In a previous post...completed about fifteen minutes ago, I blogged about doing hyper-convergence the Open Source Way.  One of the major applications where hyper-convergence is taking over is in the Virtual Desktop realm - the ability to spread load in such a way as to make a virtual desktop experience as fast as, or faster than, a physical desktop, makes it a very compelling platform.

One of the uses for Virtual Desktop is known - to me, anyway - as Application Delivery.  This really just means that, instead of delivering a full desktop experience, we're delivering a single application.  In my current role we have several situations where delivering applications is preferable to delivering full desktops.  Usually this boils down to situations where users have chosen a better desktop experience than Windows will ever hope to offer - Linux or Mac - but still need access to a couple of pesky applications that only run on Windows.  Microsoft has held out some of their key business products - Project and Visio, most notably - and there are still a few web applications that, believe it or not, actually require ActiveX controls to run properly.  I know, I know, it is very hard for me to believe, but I do live with it every day.  Add to that a particular application we have that is a .Net ClickOnce application - basically a Windows Executable delivered via a web service - and you have three or four applications that make it impossible to completely get rid of Windows.

In the past I've just used VDI to deliver a full Windows desktop to these users, but it's so much more useful at times - and quicker, not to mention cooler - to just have the application.

Enter Guacamole.  Guacamole is a clientless remote desktop gateway, and has recently been accepted into the Apache Foundation in the Incubator stage.  Essentially, Guacamole sits on a java web server (Tomcat, etc.) and proxies remote desktop sessions between a web browser and a server or desktop running a remote desktop service of some sort.  At present, Guacamole has built-in support for VNC, RDP, and SSH.  Because Guacamole is presenting a truly web-based interface, it can be presented securely over HTTPS and WSS to clients.  It also features a configurable and extensible authentication framework, with built-in support for LDAP, Database, File, and No Authentication, but with easily-available documentation for writing your own plugin.  I hope to add RADIUS to that list at some point.

Focusing in on the support for RDP, Guacamole presents a powerful interface for allowing users to connect to Windows applications within a web browser.  In Windows, you can use RemoteApp to deliver specific applications via RDP without the full Windows shell.  Guacamole supports this RemoteApp integration, as well (via the FreeRDP RAIL plugin), so you can launch a configured RemoteApp application from a Guacamole session.  In our current environment, this means running IE, .Net ClickOnce applications, and Microsoft Office tools seamlessly in a web browser.  This is particularly compelling in the case of the IE applications - using the "-k" switch in IE and passing the URL to the RemoteApp, you get a web browser (Chrome or Firefox) tab running IE, with no IE window decoration that truly looks transparent to the user.

Furthermore, the extensible authentication support means that there's the potential to make such an interface accessible outside of a VPN.  Because you're just passing HTTPS traffic, and you could presumably integrate with RADIUS or some other multi-factor authentication system, you can securely provide a way for users to access applications and desktops without the need for a VPN client.  Furthermore, because you can control the configuration of Guacamole - hide it from the users, and control what can be configured - your risk of data loss can be almost entirely mitigated, unless someone is so intent on stealing your data that they are copying and pasting via the clipboard or taking screenshots.

The total power of this solution pulls in a few other components.  First, using oVirt, you can very easily manage the VMs you use to back this sort of configuration, dynamically scaling pools of (properly licensed) Windows VMs and automating the deployment of those to support varying workloads (hmmm...elasticity...sounds like Cloud).  Of course, you could also deploy to a public cloud scenario - AWS or (shudder) Azure - and be able to scale without worrying about the hardware - you just have to worry about what that invoice will look like next month :-).Second, an incredibly powerful TCP proxy, HAProxy, can be used to do host alive checks, aggregate hosts, and load balance between VMs in a pool.  Guacamole supports some load balancing, but I've found HAProxy to be a little more powerful and configurable.  oVirt also features a REST API that might be leveraged to control VM power states for the hosts in a pool.  A script can go look at the HAProxy data and figure out how many hosts are in use, and how many are down, and, upon reaching a certain threshold for hosts in use, execute REST API commands to start up additional hosts via oVirt.  Users can point their browsers at the Guacamole URL, and, without any plugins or client requirements outside of the HTML5 browser, connect to published Application or Desktop.

And, again, the building blocks for this are all Free & Open Source - with the exception, of course, of the licenses for Windows VDA.

Hyper-convergence the Open Source Way

I've recently started an effort to migrate some of our infrastructure - primarily VDI and Development - over to a hyper-converged architecture.  I've looked at several commercial options - some bolt-ons for existing hypervisors that aggregate storage to help create hyper-convergence (e.g. VMware VSAN), along with some of the products designed to do hyper-convergence from the ground up (e.g. Nutanix).

Some of these products are pretty compelling.  Nutanix has some fantastic features, and provides their community edition software to get you hooked.  But I love a good challenge, so I decided to see if I could build my own hyper-converged infrastructure with the following goals: reuse existing hardware, and pay nothing for the software.

I should take a moment and make sure that everyone understands that, while I'm a huge fan of Free and Open Source Software, I'm not opposed to paying for things when it makes sense.  I manage an IT department, and there are plenty of times when I make the choice to go commercial with products.  But sometimes I'm just itching for a challenge, and this was one of those times.

From a commercial products standpoint, reusing hardware is hit-or-miss.  There are a few commercial options out there that are software-only options - either installs for existing systems, or virtual appliances that use raw LUN mappings to aggregate local disk space on separate servers into a single pool available to a group of servers.  VMware's VSAN product is probably the most well-known item of this ilk, but many of the other hyper-converged "solutions" are really just getting you the last piece of that: storage aggregation.  Some suppliers, like Nutanix, aren't willing to provide you a software-only version to run on existing hardware - if you're going to move beyond the limited Community Edition, you're going to buy either their hardware or someone else's "approved" appliance.  I understand the reasoning, and I may still end up with a turn-key hardware solution, but I still felt the need to find something that would work with existing hardware for no cost.

Interestingly, the product I found doesn't even bill itself as a solution for hyper-converged architecture.  We'll cover those reasons in a minute, but a web search for "open source hyper converged" doesn't show any mentions of this product on the first page, and even page two is more of a tangential reference than it is a direct hit.

The product I settled on is oVirt.  oVirt is an open source project backed by Red Hat (and the basis for RHEV), and is billed as Virtualization Management application.  The home page says they do "manage virtual machines, storage, and virtualized networks."  I certainly understand that a tool that manages those three things does not automatically equate to a hyper-converged platform, but hear me out.  In version 3.5, I believe, they added integrated support for GlusterFS into oVirt, and there's where I think we turn the corner toward an open source product that really does hyper-convergence.

If you're not familiar with GlusterFS, it's another open source project (also backed by Red Hat), for a distributed network filesystem.  It gives you a wide variety of options for how to distribute data - striping, replication, distribution, and dispersion, and several combinations thereof - to take advantage of direct-attached storage on individual compute nodes.  It then gives you a single namespace where you can get access to that data, and provide both native GlusterFS access along with NFS (either integrated or via Ganesha).

With the integration of Gluster into oVirt, oVirt truly becomes a viable hyper-converged platform.  KVM is used as the hyper visor, networking is done via OpenvSwitch, and Gluster is the distributed storage platform.  A very user-friendly web GUI ties most of this together (there are some things you have to drop down to the command linen to do, but not many), and you can use existing hardware, with integrated storage, and eliminate the need for a SAN.

Deploying oVirt with Gluster was very easy.  I choose CentOS 7 as the base platform and installed it on 7 hardware nodes.  The first node I designated as the oVirt management node (I believe support for doing this in a VM is coming, soon), and installed the oVirt packages necessary to get the management console up and running.  After the base O/S install was done on the rest of the nodes, the oVirt management console uses SSH to connect to those nodes and deploy all of the necessary software packages, including the host agent.  Once the hosts were all connected to oVirt, I was able to use the Storage Domain creation wizard in the web management console to create a GlusterFS volume, and it was automatically attached to all of the nodes.  You do need some familiarity with Gluster, particularly what type of volume you want, stripe width, etc., but other than that it is pretty straight-forward (Gluster itself is pretty easy to set up).

My test bed for this consisted of some very, very old Dell PE1955 blades.  I also have a C5000 chassis around with 8 "sleds" that I will probably use.  I'm in the process of procuring some SSDs for those systems - the CPUs and RAM are decent for the job of running VMs for various tasks, but the I/O on a single HDD - even spread across 6 nodes with Gluster - just doesn't cut it.  Eventually I may move into the Dell FX2 chassis for hardware and continue to use oVirt to manage that platform.  Just thinking about that...

Storage is really the key differentiation in many of the hyper-converged platforms.  Most either use common hypervisors (KVM is very popular) or support bringing your own (usually with KVM or ESXi).  Most feature the same rough set of networking and virtual switching components.  But how they handle storage is really where they get their features.  Nutanix boasts the concept of highly-available storage, but couples that with "data locality" - that is, the disk for a VM is replicated across multiple nodes, but at least one of those nodes is the host on which the VM is running, and the access to the data is done completely locally, eliminating any latency due to network traffic.  Furthermore, Nutanix doesn't split up the VM disk object among multiple nodes - it replicates it across multiple nodes, but the entire VM disk is present on the system where the VM is running, and can migrate/move to other nodes in the cluster.  Many other products do not feature that notion of "data locality" - and often it is because the storage is being split below the object level, at the block level, and dispersed among available nodes to overcome I/O bottlenecks.

Gluster falls somewhere in between the sophistication of Nutanix and just striping data blindly - or, it can, depending on how you configure it.  Using a Distributed Replicated volume in Gluster you can get very close to the way Nutanix works - the entire file is written to one node and then replicated to another node (or however many other nodes needed to meet your replication configuration).  According to some anecdotes - no official documentation - in this type of configuration Gluster *should* create the disk image on the system where the I/O originates, which could be where the VM is started.  However, it comes up short in two places - first, there's no guarantee that will happen, and, based on current I/O load of that system at the time, it could be initially create somewhere else.  Second, Gluster doesn't have a mechanism to trigger movement of a file or set of data from one node to another.  So, in Nutanix, if you shutdown your VM and start it on another node, or you live migrate your VM from one node to another, Nutanix will transparently move the disk to the storage closest to that node.  Gluster (currently) will not.

Anyway, there's my initial experience with oVirt as a platform for hyper-convergence.  I think it works at a base level, and I think the potential is there for it to become a true hyper-converged platform, I just hope that the folks steering the project will realize the potential and drive it in that direction.

As a final note, oVirt has some other incredibly powerful, incredibly disruptive features that could also be game-changers in giving the incumbent virtualization giants a run for their money.  First, it features the capability to do Sysprep for Microsoft Windows, and similar functions in other operating systems, that let you deploy pools of VMs in an automated fashion.  Think VDI.  Think replacement for Citrix XenDesktop and XenApp, and VMware View and Horizon.  I've been playing with that over the past week or so, and it works - really, really well.  Second, it features the SPICE protocol, a competitor to PCoIP (what VMware licenses for their accelerated display support) and HDX (Citrix).  I don't think SPICE is as mature as these other products, and doesn't seem to support server-side GPU acceleration the way those do, but give it a few months and that may change.  Finally, it has the LDAP/AD integration, multiple domains, and multi-tenancy that make it a very powerful and highly configurable management platform for multi-tenant environments.

All for now - at least on that topic.

Saturday, September 26, 2015

Enterprise OTP

Our company recently decided to switch to two-factor authentication for our VPN connections.  Actually, we call it 2.5 factor authentication, because we're using 2 x Something You Have (Token and Certificate) and Something You Know. Anyway, when we initially went down the road of one-time passwords and tokens, we picked a large and well-known commercial token provider, who shall remain nameless.  It was an obvious choice, until some Google searching led me to an open source alternative: LinOTP.

My original reason for looking for alternative OTP solutions was to find one that supported sending tokens via SMS.  LinOTP turned up as one of these, but I quickly found that it had much broader capabilities than that.  In addition to SMS, it supports E-mail tokens, as well as a variety of Smart Phone apps (including Google Authenticator).  There are also a few hardware-based tokens supported.  With the Smart Phone apps and hardware tokens, both event-driven and time-based tokens are supported.  And, with Google Authenticator, configuring the token is as easy as scanning a QR barcode on the screen when you generate the token.

Having discovered these additional capabilities of the system, I moved quickly toward setting it and evaluating it as an option to replace the commercial solution.  Several weeks later, I'm operating my VPN using LinOTP as my authentication backend.  Here are a few more highlights of LinOTP relevant to our implementation:
  • Broad support for backend storage.  In our case we used PostgreSQL - which turned out to be an even better choice (read on!)
  • Wide variety of options for user resolvers, mainly LDAP-type back ends.  We use eDirectory, which works perfectly, but LinOTP supports Active Directory, openLDAP, and just about any other user storage mechanism.
  • Relatively simple install and configuration, basically some development libraries and Python dependencies, and it's done.
  • Integration with RADIUS via either the rlm_perl module or a special rlm_linotp2 module.  We're using rlm_perl.
  • Relatively easy HA configuration.  Since we're using Postgres for the DB backend, we're able to use Bucardo to do two-way replication of our database between two separate LinOTP servers. eDirectory of course supports replication, so we have redundant LinOTP servers for high availability.
  • A fairly complete self-service portal where users can log in and perform various actions related to their tokens, including setting PINs associated with tokens, disable a lost token, etc.
  • Interfaces are relatively intuitive - at least as intuitive as the commercial solution that we had begun implementing.
As far as the cost goes, no contest there.  The commercial solution we were implementing would have $75/user for the soft (smart phone) tokens, or $125/user for the hardware tokens.  The tokens are good for three years, so part of that cost would have been recurring over those years.  The LinOTP solution does not cost anything for soft (smart phone tokens), and the only cost for the hardware tokens is the tokens themselves, about $27 for the ones we chose, which are good for five years instead of three.

So, for anyone looking for an OTP solution, I'd suggest investigating LinOTP and see if it works for you.