midipix

write once with confidence

/pre/alpha: work hard till things just work (tm)

project news

latest commits

what is midipix, and how is it different?

midipix is a development environment that lets you create programs for Windows using the standard C and POSIX APIs. No compromises made, no shortcuts taken.

If you are interested in cross-platform programming that reclaims the notion of write once, compile everywhere; if you believe that the 'standard' in the C Standard Library should not be a null signifier; and if you like cooking your code without #ifdef hell and low-level minutiae, then this page is for you.

midipix makes cross-platform programming better, simpler and faster, specifically by bringing a modern, conforming C Runtime Library to the Windows platform. While the idea itself is not new, the approach taken in midipix to code portability is radically different from that found in other projects.

layering and modularity
midipix fosters true separation between the POSIX system call layer and the C runtime library. In other projects that provide a POSIX programming environment, such as cygwin and SUA/Interix, there is no practical way to tell the (2) from the (3), and one often finds that the system call interface, e.g. poll(2), is unified with its respective libc function. Although closely related, C and POSIX are two independent standards that evolve differently and at a different pace. Between the two, midipix focuses on the part that is painfully missing on Windows, namely the POSIX system call layer.
complete, fast, and robust
The midipix system call layer is written using the Windows Native API, this including the implementation of sockets, pseudo-terminal file descriptors, job control, path resolution, inter-process communication, and virtual mount system. The use of the Native API provides key performance increases, most notably when compared with cygwin, and the completeness of the system call layer (see roadmap) provides a true advantage over msys and mingw. Additional features that distinguish midipix from cygwin and mingw include a copy-on-write fork, the native implementation of thread- and process-creation, the native handling of utf-8 in system call arguments, and the use of fast LPC for signals, session-management, and locking arbitration.
easy distribution
midipix supports both static and dynamic linking; there are no assumptions about the directory structure on the end-user machine, and unlike SUA/Interix, no special installation of a Windows component is required. Midipix applications running on the same system are completely independent of one another; as opposed to cygwin there is no arbitrary inherited state between two unrelated applications, and the crashing or misbehavior of one application can in no direct way affect another. The one intentional exception to the rule is that of POSIX sessions, where you would want such inheritance and integration to take place. For instance, you would want an application created via fork(), execve(), or posix_spawn() to inherit its file descriptors (and in the case of fork() also address space) from its parent, and would likewise want termination of a child application to cause a SIGCHLD signal to be sent to the parent. In all other cases, however, midipix applications would only interact with one another if you so desire (using your favorite method of IPC), thus making both distribution and debugging considerably easier.
flexible execution environment
midipix applications are part of the native Windows subsystem; unlike SUA/Interix there is no need for special kernel support, yet double-clicking a midipix application will never open a terminal window against your will, as in the case of msys or cygwin. At the same time, and similar to other unix-like systems, midipix applications that are started from within a terminal session can automatically take advantage of job control, (pseudo-)terminal signals, and standard input and output.
a modern libc of your choice
Whereas cygwin is paired with newlib, mingw depends on msvcrt, and SUA mandates its own C library, the midipix system call layer (psxscl) is not tied to any particular libc, and may thus be used for porting several different implementations. The midipix libc of choice, however, is musl: a modern libc implementation that puts special emphasis on portability and correctness, and which provides complete, high-quality, and exceptionally robust pthread- and C11 support. The musl developers and users in general, and the project architect Rich Felker in particular, have been tremendously helpful in shaping the direction of the midipix project, and have provided invaluable tips, suggestions, and technical feedback. A tiny set of files that bridge between an application's entry point and musl's __libc_start_main, the midipix musl-glue files turn NT into just another supported system, and may also be used as a model for porting additional libc's to Windows.
portability from below
Rather than porting a POSIX C library by way of modification, midipix renders porting easy by satisfying the libc's largest (and often only) dependency, namely the system call layer. In doing so, midipix follows the design of musl libc, where about 99% of the code base is shared between platforms, and where the few low-level, architecture-specific bits remain transparent to application and library authors. There are several clear advantages to that approach, most notably with respect to sustainability and reliability; sustainability, since maintaining the system call layer does not require maintaining a libc on top of it; and reliability, since every single function in the libc gets tested and reviewed by a large number of users in a variety of platforms. Then again, there is virtually no lag between the introduction of a feature to the libc, and its availability on Windows, and it is likewise easy to switch between different version of the C library for testing purposes.

implementation details

unicode as expected
Unicode in midipix is not an afterthought, but rather an essential concept. When your application enters its main() function, for instance, its argv and envp point to utf-8 strings, just like they would on modern unix systems. Similarly, the size of the framework's native wide character (wchar_t) is four bytes, meaning that your unicode application can share its entire code-path between platforms. Last but not least, midipix adds an important 'M' (as in multibyte) variant to the 'A' (ansi) and 'W' (utf-16) GUI flavors, meaning that you can seamlessly call functions such as GetWindowText or SetWindowText using utf-8 input and output.
flexible path resolution algorithm
The system call layer supports both POSIX (/c/midipix) and DOS (C:\midipix) notations, as well as the operating system's native (\??\C:\midipix) path syntax (the term 'native path' refers here to a utf-16 path buffer that may be passed to, or returned from the kernel; for further reading, see the Open Specifications in general, and the SMB-2 document in particular), functions such as open or fopen, you may pass a utf-8 argument using any of the above notations, and may even combine POSIX and DOS notation in one and the same argument. To understand why this is useful, consider a scenario where you need to concatenate a path received from a third-party -- be it an application, a library, or input from the user -- with a sub-directory in POSIX notation. With C:\some\path being the inherited path, and /sub/directory/name being the addition, midipix will resolve C:\some\path/sub/directory/name by treating the DOS portion the way explorer would (including native device mounts and junction points), then proceeding with the POSIX portion. If the user started the application from within a terminal session and has mounted another folder at sub/directory/name, or had created 'name' as a symbolic link to another folder, then the mount and/or symbolic link will likewise be respected.
flexible utilization of system components and third-party libraries
In addition to providing utf-8 support for the basic GUI libraries, midipix is both accommodating and flexible with respect to other components you might want to utilize. As an example, you may implement the logic portion of your application using any number of cross-platform libraries, yet use GDI, wxWidgets, or Shell Interfaces for graphical front-end or interaction with the desktop. As part of accommodating this scenario, midipix can convert "foreign" threads (threads not created by pthread_create) into pthreads on the fly, meaning that there is no limitation on which callback functions may be invoked by third-party libraries. As should be noted, this conversion is not only very fast, but also conditional; it does take place only once, upon the first invocation of pthread_self(), and entails absolutely no overhead otherwise.
easy porting
Allowing the vast majority of popular projects to use musl libc on Windows should normally be as easy as adding midipix to the list of unix- and linux-like platforms in a project's configuration file. In the event that your project contains conditional code paths for different GUI front-ends (X11, GDI, etc.), you might need to ensure that your #ifdef's do not conflate the operating system with the graphical user interface, in other words, that you distinguish between the OS macro WIN32, and the various GUI macros such as GDI_ENABLED, WAYLAND_ENABLED, X11_ENABLED, and similar. Given the growing number of graphical user interfaces, however, making the above distinction is likely to benefit your project regardless of midipix, and will surely make it easier to maintain in the long run.
open source from the first release
The project has already entered its advanced internal development stage. Source repositories of the various runtime components are now public, and may be used under either the GPLv2 or GPLv3. To view the most recent changes, please join the project's freenode irc channel (#midipix) and ask for the address of the internal repositories.
In addition to the runtime components, the project provides several cross-platform tools and libraries that were all written from scratch, most notably slibtool, a libtool drop-in replacement, and perk, a PE Resource Kit library with an accompanying command-line tool). Midipix is a long-term, fully professional project, which considers micro-contributions from a large number of users to be the preferable form of financial viability. As an additional incentive for users to support midipix, the project will be providing digitally signed toolchain and utility binaries, thus saving you the hassle of having to build everything from source, should you prefer that option.

roadmap

15 March 2015
release of the midipix gcc port and cross-compiler, along with the libc glue files.
5 April 2015
release of the midipix Native API and free-standing environment libraries (dalist, pemagine).
27 July 2015
release of an initial version of the system call layer library (psxscl), the Native API core library (ntapi), and the extensible subsystem and pty server (ntctty).
August 2015
debugging, work towards a pre-alpha release.
September-October 2015
internal release: including the midipix cross-compiler as well as a native toolchain, the core shell utilities, and selected test applications. Terminal sessions at this internal stage use a cygwin mintty in raw mode which then invokes ntctty. This is rather cumbersome, thus making a native midipix port of minttty a high priority. For additional information, please join the project's freenode irc channel (#midipix).
November 2015 - January 2016
debugging, work on tools needed by the WINAPI headers and midipix-specific import libraries. see project news for more information.
February-April 2016
preparation for alpha release: including a native port of mintty and a nice collection of third-party utilities, libraries, and applications. pre-alpha binaries and tarballs are generated by the recently added midipix_build.sh, and can be downloaded either as a tarball or individually. For additional information, please join the project's freenode irc channel (#midipix).
May-August 2016
preparation for alpha release: the final stretch! The midipix_build project on github has already reached a fairly mature stage, and can be used to generate the alpha release tarballs (remember to ask for the address of the ntctty and psxscl repositories on irc). The ported applications provide excellent test cases for debugging, and are also much fun to use (when they work). Most of the native development is already based on the above ported tools, and the intent is to be entirely self-sufficient (most notably with respect to gdb, rsync, scp, git, curl, and a terminal emulator) by the time of the official alpha release. For additional information, please join the project's freenode irc channel (#midipix).