The Way of the great learning involves manifesting virtue, renovating the people, and abiding by the highest good.

2009年3月24日星期二

Why don't we rewrite the Linux kernel in C++?

  • (ADB) Again, this has to do with practical and theoretical reasons. On the practical side, when Linux got started gcc didn't have an efficient C++ implementation, and some people would argue that even today it doesn't. Also there are many more C programmers than C++ programmers around. On theoretical grounds, examples of OS's implemented in Object Oriented languages are rare (Java-OS and Oberon System 3 come to mind), and the advantages of this approach are not quite clear cut (for OS design, that is; for GUI implementation KDE is a good example that C++ beats plain C any day).
  • (REW) In the dark old days, in the time that most of you hadn't even heard of the word "Linux", the kernel was once modified to be compiled under g++. That lasted for a few revisions. People complained about the performance drop. It turned out that compiling a piece of C code with g++ would give you worse code. It shouldn't have made a difference, but it did. Been there, done that.
  • (REG) Today (Nov-2000), people claim that compiler technology has improved so that g++ is not longer a worse compiler than gcc, and so feel this issue should be revisited. In fact, there are five issues. These are:
    • Should the kernel use object-oriented programming techniques? Actually, it already does. The VFS (Virtual Filesystem Switch) is a prime example of object-oriented programming techniques. There are objects with public and private data, methods and inheritance. This just happens to be written in C. Another example of object-oriented programming is Xt (the X Intrinsics Toolkit), also written in C. What's important about object-oriented programming is the techniques, not the languages used.
    • Should the kernel be rewritten in C++? This is likely to be a very bad idea. It would require a very large amount of work to rewrite the kernel (it's a large piece of code). There is no point in just compiling the kernel with g++ and writing the odd function in C++, this would just result in a confusing mix of C and C++ code. Either the kernel is left in C, or it's all moved to C++.
      To justify the enormous effort in rewriting the kernel in C++, significant gains would need to be demonstrated. The onus is clearly on whoever wants to push the rewrite to C++ to show such gains.
    • Is it a good idea to write a new driver in C++? The short answer is no, because there isn't any support for C++ drivers in the kernel.
    • Why not add a C++ interface layer to the kernel to support C++ drivers? The short answer is why bother, since there aren't any C++ drivers for Linux. However, if you are bold enough to consider writing a driver in C++ and a support layer, be aware that this is unlikely to be well received in the community. Most of the kernel developers are unconvinced of the merits of C++ in general, and consider C++ to generate bloated code. Also, it would result in a confusing mix of C and C++ code in the kernel. Any C++ code in the kernel would be a second-class citizen, as it would be ignored by most kernel developers when changes to internal interfaces are made. A C++ support layer would be frequently be broken by such changes (as whoever is making the changes would probably not bother fixing the C++ code to match), and thus would require a strong commitment from someone to regularly maintain it.
    • Can we make the kernel headers C++-friendly? This is the first step required for supporting C++ drivers, and on the face seems quite reasonable (it is not a C++ support layer). This has the problem that C++ reserves keywords which are valid variable or field names in C (such as private and new). Thus, C++ is not 100% backwards compatible with C. In effect, the C++ standards bodies would be dictating what variable names we're allowed to have. From past behaviour, the C++ standards people have not shown a commitment to 100% backwards compatibility. The fear is that C++ will continue to expand its claim on the namespace. This would generate an ongoing maintenance burden on the kernel developers.
      Note that someone once submitted a patch which performed this "cleaning up". It was ~250 kB in size, and was quite invasive. The patch did not generate much enthusiasm.
      Apparently, someone has had the temerity to label the above paragraph as "a bit fuddy". So Erik Mouw did a short back-of-the-envelope calculation to show that searching the kernel sources for possible C++ keywords is a nightmare. Here is his calculation and comments (dates April, 2002):
      % find /usr/src/linux-2.4.19-pre3-rmap12h -name "*.[chS]" |\     xargs cat | wc -l   4078662  So there's over 4 million lines of kernel source. Let's assume 10% is comments, so there's about 3.6 million lines left. Each of those lines has to be checked for C++ keywords. Assume that you can do about 5 seconds per line (very optimistic), work 24 hours per day, and 7 days a week:                   5 s   1 hour     1 day   1 week 3600000 lines * ------ * -------- * ---------- * -------- = 29.8 weeks                  line     3600 s     24 hours     7 days  Sounds like a nightmare to me. You can automate large parts of this, but you'll need to write a *very* intelligent search-and-replace tool for that. Better use that time in a more efficient way by learning C. 
      Note that this is the time required to do a proper manual audit of the code. You could cheat and forgo the auditing process, and instead just compile with C++ and fix all compiler errors, figuring that the compiler can do most of the work. This would still be a major effort, and has the problem that there may be uses of some C++ keywords which don't generate a compiler error, but do generate unintended code. In other words, introduced bugs. That is not a risk the kernel development community is prepared to take.

    My personal view is that C++ has its merits, and makes object-oriented programming easier. However, it is a more complex language and is less mature than C. The greatest danger with C++ is in fact its power. It seduces the programmer, making it much easier to write bloatware. The kernel is a critical piece of code, and must be lean and fast. We cannot afford bloat. I think it is fair to say that it takes more skill to write efficient C++ code than C code. Not every contributer to the linux kernel is an uber-guru, and thus will not know the various tricks and traps for producing efficient C++ code.
  • (REG) Finally, while Linus maintains the development kernel, he is the one who makes the final call. In case there are any doubts on what his opinion is, here is what he said in 2004:

    In fact, in Linux we did try C++ once already, back in 1992.

    It sucks. Trust me - writing kernel code in C++ is a BLOODY STUPID IDEA.

    The fact is, C++ compilers are not trustworthy. They were even worse in 1992, but some fundamental facts haven't changed:

    • the whole C++ exception handling thing is fundamentally broken. It's _especially_ broken for kernels.
    • any compiler or language that likes to hide things like memory allocations behind your back just isn't a good choice for a kernel.
    • you can write object-oriented code (useful for filesystems etc) in C, _without_ the crap that is C++.
    In general, I'd say that anybody who designs his kernel modules for C++ is either
    • (a) looking for problems
    • (b) a C++ bigot that can't see what he is writing is really just C anyway
    • (c) was given an assignment in CS class to do so.
    Feel free to make up (d). 

Oberon (operating system)

Oberon is an operating system, originally developed as part of the NS32032-based Ceres workstation project; it is written entirely in the Oberon programming language. The basic system was designed and implemented by a team of two part-time programmers at ETH Zürich (ETHZ), Niklaus Wirth and Jürg Gutknecht. It was later extended and ported to other hardware by a team at ETHZ. For a full listing of team members, see ETH Oberon Hall of Fame.

Oberon has a text user interface or TUI. It combines the point-and-click convenience of a graphical user interface (GUI) with the linguistic strength of a command line interface (CLI) and is closely tied to naming conventions of the Oberon language. Any text appearing on the screen can be edited and used as command input. No prompt or anything similar is required. Although radical, the TUI is efficient and powerful.[citation needed] It has yet to appear in more commonplace operating systems, although strongly inspired Rob Pike's Acme system under Plan 9.

The Oberon OS is available for several other hardware platforms, generally in no cost versions. It is typically extremely compact. Even with an Oberon compiler, assorted utilities including a web browser, TCP/IP networking, and a GUI, the entire package has been able to fit on a single 3.5" floppy disk. The version which runs on bare PC hardware is called Native Oberon.

There is also a version called Oberon V4 that is closer to the original operating system developed by N. Wirth. It was also developed at ETHZ, but the most recent version is at Linz university. Oberon V4 appears to be orphaned, there are almost no changes since 2000.

The computer science department at ETHZ has in recent years begun exploring active objects, and concurrency for operating systems and has released an early version of a new language Active Object Oberon and a new operating system for it, first called AOS and now called A2 and/or Bluebottle. It is available from ETHZ with most source via the Internet. Versions are currently available for Intel IA-32 single and multi processor systems and for the StrongARM CPU family.

2009年3月22日星期日

Google Summer of Code 2009 ideas

Want to spend your summer contributing full-time to Gentoo, and get paid for it? Gentoo is applying for its 4th year in the Google Summer of Code. In the past, almost half of our successful students have become Gentoo developers, so your chances of becoming one are very good if you're accepted into this program.

Contents

 [hide]

[edit]For Students

Most ideas listed here have a contact person associated with them. Please get in touch with them earlier rather than later to develop your idea into a complete application. You can find many of them on Freenode's IRC network under the same username. If there is no contact information, please join the gentoo-soc mailing list or #gentoo-soc on the Freenode IRC network, and we will work with you to find a mentor and discuss your idea.

You don't have to apply for one of these ideas! You can come up with your own, and as long as it fits into Gentoo, we'll be happy to work with you to develop it. Remember, your project needs to have deliverables in less than 3 months of work in most cases. Be ambitious but not too ambitious ;)

We will know whether Gentoo is accepted by 18 March, and you can apply starting March 23 -- the earlier the better! We have a custom application template that we will ask you to fill out. Here it is:

Congratulations on applying for a project with Gentoo! To improve your chances of succeeding with this project, we want to make sure you're sufficiently prepared to invest a full summer's worth of time on it. In addition to the usual application, there are 2 specific actions we would like to see from you:

  • Use the tools that you will use in your project to make changes to code (e.g., source code management [SCM] software such as CVS, Subversion, or git). Please use the same SCM as you will use for your project to check out one of our repositories, make a change to it, and post that change as apatch on a mailing list or bug. You don't necessarily have to fix any real bugs; this is to show that you can use the tools. Your contact in Gentoo can help you determine which SCM and repository you should use for this. If your idea doesn't have a contact, please get in touch with us on the gentoo-soc mailing list or in real-time on IRC at Freenode/#gentoo-soc. Once you've made your change, link to it from your application.
  • Participate in our development community. Please make a post to one of our mailing lists and link to it from your application (archives.gentoo.org holds past postings). The gentoo-soc list would be a good starting point, if you aren't subscribed to any others already.

Both of these actions are things you will do extremely commonly as an open-source developer, and they really aren't that hard, so don't let them hold you back! The remainder of the application is free-form. Please read our application guidelines and Google's FAQ to complete it. Good luck!

[edit]Ideas

[edit]Add "tags" support to Portage

Gentoo uses categories now. A package can only be in a single category, which is very limiting because generally things don't fit perfectly into one place without other possibilities. Tags could make it a lot easier to find packages they're looking for by doing Boolean searches like: kde AND mail. This project would add support for tags to Portage and would allow for backwards compatibility of categories as tags.

Skills:

  • Python

Contacts:

[edit]Portage/ebuild ability to use file-based capabilities rather than setuid

With recent Linux kernels, file-based capabilities are available. It is thus possible to give the ping command just the minimum capabilities needed to access the RAW socket, rather than leaving it entirely setuid. A long term goal for Gentoo would be to allow the user to choose capabilities over setuid for (at least some) programs. To support this feature however, Portage needs a means to copy over capabilities from one file to another. A python extension may be needed to handle this.

Skills:

  • Python

Contacts:

[edit]Fastboot on Gentoo

A few months ago, some kernel developers working at Intel got boot times on solid-state drives down to 5 seconds (10 on hard drive). It would be awesome to have this in Gentoo. This would require significant changes to our custom init system, OpenRC, as well as lots of profiling of the kernel boot and userspace boot processes using tools like bootchartMore info about fastboot is here. Note that the work of fastboot was primarily concerned with:

  1. sreadahead, for which there already are packages in the tree, and
  2. optimizing the init system (which might be a bit tough for OpenRC).

Skills:

  • Shell scripting, C

Contacts:

[edit]Port the new distro-neutral initrd framework, Dracut, to Gentoo

Instead of every distro having its own way to generate an initramfs, many people would like this to ship with the kernel. It's developed primarily by Red Hat with a fresh start and thought put into portability. See this LWN article for more info.

Skills:

  • Shell scripting

Contacts:

[edit]Make the clustering LiveCD from last year's GSoC bootable from USB

The clustering LiveCD is a bootable image that allows you to instantly turn a full room of computers into a functional cluster. You only need one CD because the rest of the nodes boot disklessly from the master node with the CD. This project would be a huge improvement to the existing version because it will allow us to use persistent, writable USB media instead of a read-only CD. That means clusters won't need to be reconfigured from scratch on every boot. That will save a lot of admin time on each boot and make it possible to do things like run a cluster in 10 different computer labs with different settings on each, as easily as inserting 1 USB stick in each lab and rebooting all the computers.

Skills:

  • Shell scripting, maybe some Python

Contacts:

[edit]BSD ports of Gentoo: OpenBSD, NetBSD, DragonFlyBSD, etc.

We've never had a DragonFlyBSD port, and the OpenBSD and NetBSD ports are dead. This project would involve investigating and implementing the necessary porting work to get a Gentoo userland running on a BSD kernel and base system (the base system is also managed by Gentoo).

Skills:

  • Shell scripting, C (patch creation)

Contacts:

[edit]EBuild generator

For the basics, writing an ebuild could be some pythonish gui thing to get stuff like dependencies and some common code blocks in there. Plus it's easier if you don't need any special overrides. One attempt to create such a tool was Abeni.

Skills:

  • Python

Contacts:

  • ? (Your name here)

[edit]Portage/Pkgcore/Paludis backend adapter for PackageKit

PackageKit is a UI for getting packages installed. Currently there is no backend support for Gentoo-based package managers. This project would consist of authoring the correct adapters to allow PackageKit to utilize one or more existing Gentoo package managers such that users could use PackageKit on Gentoo.

Skills:

  • Python

Contacts:

[edit]Cache sync

The portage tree and all its overlays keep growing. Right now only the official portage tree occupies more than 600Mb on a regular filesystem. However the package manager does not need the whole tree of full ebuilds, patches and manifests to perform most of its work. The idea would be to sync a smaller database or a cache of only needed information for global package manager operations, then fetch the required package only when needed. It would speed considerably tree synchronization and reduce the space occupied by portage tree. Currently the cache system in portage is also really slow and so is the search feature. The project could be inspired by the Debian or RPM system but with the usability and choices offered by Gentoo, and would probably include:

  • design and implement automatic cache builder to be produced by a given repository/overlay
  • make portage/paludis/pkgcore to do delta-sync with a local cache and fetch only the required files to be installed when requested

Skills:

  • Python (portage/pkgcore) or C++ (paludis)

Contacts:

  • Your name here.

[edit]Improved binary package support

Portage lacks a few features that would make binary package support much smoother and less prone to breakage, which would make Gentoo better for derived binary distros. One of them is more intelligent handling of library versions with binpkgs. Basically, it's possible to build a binpkg against an old version of a library, then install it against a new version and have it be broken by default because of a shared-library version bump. Ideally, a package would have a way to specify which exact files it depends in the built state on instead of just which versions it can build against from source. Another problem is saving binpkgs with different USE flag and other build settings on the same host. See bug #150031. The way forward is one or more hashes of the metadata. A third problem is the lack of binpkg support for the kernel. This could be changed through modifying the kernel eclass to support a binary USE flag that also did configuration & build, or perhaps some kind of genkernel modification, or both.

Skills:

  • Python, shell scripting

Contacts:

[edit]Tools and support for multiple FORTRAN compilers

The FORTRAN language is unfortunately not dead yet. Many scientists still use it and your pet program might need it. Gentoo has a few FORTRAN compilers, but the framework is getting old and could be much improved. Some powerful compilers are not allowed. The task could include rewriting the framework to allow any user compiler, FORTRAN profiles with eselect, writing documentation, testing applications, making benchmark on linear algebra. Basically making FORTRAN on Gentoo more configurable, robust and easy.

Skills:

  • Shell scripting, Fortran

Contacts:

[edit]Create and release a Gentoo stats server/client

A few Gentoo stats projects have happened in the past, including

But right now, we have nothing in a usable state. The one from GSoC 2006 may have come the closest, but it ran into some major issues with authentication and security. Having stats available would be a huge benefit to Gentoo developers (we would know how important packages are to users), to upstream package developers (who would know how well-tested various versions are), and to show people how popular Gentoo is.

One intriguing option is to port Smolt to Gentoo. Smolt is a package written by Red Hat that is intended to be distribution-neutral. Here's an example of the stats. As part of this project, it would be great to enhance Smolt so that it also reported installed packages in addition to hardware.

Skills:

  • Likely Python and/or shell scripting. Possibly some PHP for a webapp.

Contacts:

[edit]Write G-PEAR (inspired by G-CPAN)

This project involves writing a tool (tentatively named G-PEAR) that generates and installs PEAR packages on the fly. The tool could closely emulate the structure of G-CPAN, that is open to discussion however.

Skills:

  • Likely PHP

Contacts:

[edit]Create a Web-based Gentoo image builder

It would be awesome if people could go to a website and build a custom Gentoo image. This project would entail writing a webapp (along the lines of rPath rBuilderSUSE Studio, or Angstrom Narcissus). On the backend, it might use a tool like Gentoo's release-builder, Catalyst. This may entail modifications to Catalyst in addition to writing the webapp from scratch. To be successful on this project, you will have to be a very driven, independent student with web development experience, as our potential mentors in this area are extremely busy.

Skills:

  • PHP and/or Python

Contacts:

[edit]Tree-wide collision checking and provided files database

File collisions occur in Portage when a package tries to overwrite a file already installed by another package. Collisions are QA issues (packages involved in a collision need to either block each other or resolve the conflict) and are reported as QA warnings/errors by emerge, but only occur if the offending package is installed on the system. Because of the size of the Portage tree, nobody tests for collisions with all packages.

This project would develop a tool to provide such a test. This would involve setting up a tinderbox (which would eventually become part of the Gentoo infrastructure) to install as many packages from the Portage tree and registered overlays as possible, and prompt the maintainer to install packages which failed to emerge automatically. A database of package contents would be compiled and exposed (read-only) on a server. To perform the check on a new package, a client-side utility would either download the database and check locally, or submit the package's contents for checking to the server. Also, the tinderbox would log all collisions it encountered when installing the packages already in the tree, and add any new packages from the trees nightly.

In addition, the server would provide a Web interface for a user to query a filename and see which packages provide that file.

USE flag changes and live ebuilds can affect package contents, so this would not be a comprehensive test, but it would catch most collisions.

Skills:

  • Likely Python and shell scripting

Contacts:

[edit]SCM snapshot management infrastructure

A growing number of projects provide binary-only releases of their packages with the source available only as a tag in a version control system. This presents obvious problems for source-based distributions, as we must provide one of two possible solutions:

  • live ebuilds, which pull the tag directly, making them vulnerable to a single point of failure (the upstream SCM server) and the possibility of upstream altering the tagged contents, as well as possibly placing an excess load on the upstream servers; or
  • snapshots, which must be pulled and manually packaged by a developer and hosted on Gentoo infrastructure. Snapshot archives are often not reproducible bit-for-bit and do not have upstream's direct "blessing".

This project would implement a Gentoo snapshot management infrastructure (an extension of Gentoo's existing mirror infrastructure) to provide a better alternative. The process would be:

  • The ebuild writer specifies a scm url and tag
  • If the writer has no access (the ebuild is not uploaded to main tree or a listed overlay), the ebuild behaves as a live ebuild.
  • Otherwise, the snapshot manager daemon packages the snapshot and posts it on Gentoo mirrors. The ebuild fetches the snapshot and uses it. (this will require a manifest update upon/after commit...)
  • The snapshot manager daemon periodically checks the sources of all snapshots for changes that would alter the contents of the snapshot. It alerts the ebuilds' authors to any such changes.

The coding part would involve writing the aforementioned daemon. The mentor would have to be someone from infra, or at least the student would be interacting with infra a lot.

Skills:

  • Likely Python or shell scripting

Contacts:

  • ? (Your name here)

[edit]Adapt Kuroo for current portage versions

Kuroo made using Gentoo feel easy - just as it should. Finding and installing packages was very easy, as were most common maintenance tasks, like unmasking a testing ebuild or adjusting USE flags.

Sadly the Kuroo backend couldn't keep up with the pace of development of portage (for lack of time), so it doesn't work anymore with current portage versions.

Getting it working again and future proof would regain a great package manager GUI for Gentoo, opening Gentoo up for many new users, especially for people who had someone else install their Gentoo for them and now need a good GUI to maintain their Gentoo.

This project would involve:

  • Porting Kuroo to Portage >=2.1 and KDE 4
  • Finding and implementing a way to make it easy to maintain during future portage changes.

Skills:

  • Python

Contacts:

  • ? (Your name here)

[edit]Octave/Matlab support

Octave support could be really improved in Gentoo. Octave has now a core+plugin architecture. In order to facilitate the inclusion of octave-forge packages, one step would be to create an ebuild installer similarly what is done for perl g-cpan G-CPAN, but it could also use paludis infrastructure. Also there is still open work to integrate matlab programs compatible with octave such that one could use matlab or octave seamlessly. Some work was done by markusle last year on the science overlay the project could rely upon.

Skills:

  • Likely shell scripting or possibly Python, C++ if paludis

Contacts:

[edit]R packages ebuild generator or installer

Gentoo R users have to rely on R installer to install CRAN packages. The project would involve using Gentoo package manager to install one of the few thousands R packages. Paludis used to have CRAN support that the project could be based on, but doing a g-cpan like tool to generate ebuild would also be good.

Skills:

  • Likely shell scripting or possibly Python, C++ if paludis

Contacts:

[edit]Universal select tool

eselect is a versatile tool to switch between versions, implementations or in Gentoo. However eselect is not maintained anymore. The project would involve either give eselect some love or start new one. One very useful feature apart from consistency and all the eselect nice stuff it already offers, would be to make it not only sysadmin but also user friendly. A user could easy switch between gcc versions, intel compilers, python versions, etc...Some features are found in the module tool are not in eselect and vice-versa. It would be particularly useful in a common server and clusters to have a universal, robust and consistent tool for switching versions or implementations.

There's eclectic in Exherbo so probably should just switch to that. In that case not enough work here for GSoC.

Looking at the amount of code involved in modules, there's still many features eclectic could add to match. Making eselect/eclectic for compilers is also desirable. Looking at gcc-config sources, it would need some coding translating it to eselect.

Skills:

  • Shell scripting

Contacts:

  • ? (Your name here)

[edit]Gemtoo: A Gentoo-specific 'gem install' replacement

Gentoo currently uses the gem installer provided by rubygems to install Ruby gems. The project involves writing a new gem installer in Ruby specific to Gentoo, leveraging as much as possible the existing gem code. This makes it possible to use the normal gems to install software, but much more in sync with Gentoo's phases such as unpack, configure, compile, test, and install. This in turn allows easy patching, QA checks, and running of tests for gems when installing.

Skills:

  • Ruby

Contacts: