basement blog

Alright, progress report number 4, here we go.
Here is some terminology you should be familiar with before reading about this project. If this if you first time reading this blog:
Abstract syntax tree: AST Info – The links at the bottom are good.
The project homepage as well as the project overview posted my first week.
Current status

This is a list of things that I promised to implement over the summer to pass the GsoC project.

• Augeas grammars for most configuration files on a standard unix system.
• Displaying important nodes in an AST
• Writing ASTs back to files
• Displaying diff generated from ventoo
• Remote configuration of one other computer at a time
• rc-update integration
• Parse existing documentation form config files, displayed in embedded html browser
• Context aware custom documentation
• Smart completion of possible config values.

This is a list of things that I’ve added to my proposal informally, time permitting

• Ventoo ebuilds
• Add some functions to Augeas I could use (abandoned, found a work around)
• Remote editing of multiple computers (a lot of work, no promises here)
• User and developer manuals for Ventoo

Items in green are finished. Items in red have not been started. The item in orange is in progress.

I've spent that last couple of weeks working on making things stable. This week my development computer went down so I had to build a new one, so that took the better part of this week. Everything is back up and running. Emacs is configured, git is installed, hardware is running. Regardless, I'm ready for broader distribution of Ventoo. I'm asking anybody who is interested to add the ebuild located here to their local overlay and give it a whirl. If anything goes wrong or doesn't work as expected, let me know and send me the offending config file so I can take a look at it. Before you run the program read the user manual. It is completely up to date at the time of this writing. You can send bug reports to

Hopefully I didn't make that too hard to read. Make sure you put VENTOO in caps in the subject so I'll be sure to see it.

I'm working on passing my midterm evaluations. I hope that with many people trying to break my program it will be come very dependable and very stable. Once that is done I can use the rest of the summer to add smart completion, and remote editing. There is also some administrative stuff to do, like a help menu, about dialog, and maybe a splash screen. If I finish that early then I can just add more and more modules to be able to work with more and more files.

This is the first of several progress reports for each week of GSoC. This is basically a glorified change log as well as a manual for using ventoo in its current state. At the end there is a section on the short term project direction as well as difficulties I may be having. I put images and code into this post, and I’m not sure how it will look on an RSS feed, so this page may be best viewed on the page I’m hosting.

Current status

This is a list of things that I promised to implement over the summer to pass the GsoC project.

• Augeas grammars for most configuration files on a standard unix system.
• Displaying important nodes in an AST
• Writing ASTs back to files
• Displaying diff generated from ventoo
• Remote configuration of one other computer at a time
• rc-update integration
• Working documentation displayed in embedded html browser
• Optional custom documentation

This is a list of things that I’ve added to my proposal informally, time permitting

• Ventoo ebuilds
• Add some functions to Augeas I could use (this is difficult and I may not finish)
• Remote editing of multiple computers (a lot of work, no promises here)
• User and developer manuals for Ventoo
  

Items in green are done. Items in red have not been started. The items in orange are almost done or in progress.

I’ve also started a user manual. It has a temporary home on gentoo soc hosting.

Current Difficulties

Things are going pretty smoothly. I got some feedback about remote file editing from my last post, so that will surely help out when the time comes. If you happen to get a copy of the code right now, there is a pretty serious bug. If you point Ventoo at your root directory, which is the default, it won’t display the AST properly. You should run these commands to make a copy of /etc then edit that instead. cp -R /etc /tmp then python testgui.py /tmp works for me. I’m guessing I made a mistake joining paths or stripping slashes somewhere.

Short term direction

Right now I’m working on writing Ventoo modules. Augeas can parse many files out of the box on gentoo, but I’m going to try to write “lenses” for more gentoo specific files like packages.use, keywords and make.conf.

Here are some examples of Ventoo module files.

The mult attributes define how many times a node can appear in an AST. Right now ventoo understands 0-9, +, ?, and *.

/etc/fstab

<VentooModule>
  <root mult="*">
    <spec mult="1"/>
    <file mult="1"/>
    <vfstype mult="1"/>
    <opt mult="*"/>
    <dump mult="1"/>
    <passno mult="1"/>
  </root>
</VentooModule>

/etc/groups
<VentooModule>
  <root mult="1">
   <ventoo_dynamic mult="*">
     <password mult="1"/>
     <gid mult="1"/>
     <user mult="*"/>
   </ventoo_dynamic>
 </root>
</VentooModule>

Ventoo_dynamic catches all nodes (except comments) and displays them. This is used in /etc/groups because group names are user defined and Augeas can’t know them ahead of time.

Screenshots

Everybody loves screenshots.

Here are two screenshots of editing the file /etc/hosts. Note that Ventoo understands that there can be many address entries, and that each entry can have exactly one ipaddr and canonical entry, but unlimited aliases. These multiplicities are defined in the Ventoo module.

I haven’t started an official git repository yet, but it will appear on the temporary project webpage when I do create one.

This summer I’ll be developing the “Ventoo” project.

Ventoo is a mix of Gentoo and “Visual configuration editor”.

Project overview

At its heart, Ventoo takes config files like /etc/hosts,  /usr/portage/packages.keywords    and turns them into abstract syntax trees (ASTs) then lets the Ventoo user edit these trees and save them back into the original file with minimal changes to the original text. Eg, comments and white spaces are preserved.

Ventoo is written in python and uses Augeas to parse the text files into configuration files. pyGTK is used for widgets. The main widget for the whole program is a treeview widget that displays the AST for each parsed file. Once changes have been made, Ventoo creates two versions of the file, the original and the modified one, and then compares them in a user defined diff viewer for merging. I find Meld works well for this.

Ventoo, with the help of Augeas, user defined regular expressions, and python scripts can do a pretty good job of validating and providing help as well as official documentation to the user on the fly.

This is fine and dandy, but ultimately I find it only slightly better than just editing the files in emacs or similar. It can also be thought of as training wheels for unfamiliar config files.  I’m actually building Ventoo so that it will be more of a LAN administration tool.

Why you should care

If you’re the type that doesn’t mind editing config files by hand (and I assume that is the case for most people, including myself) then chances are you’re not going to think of Ventoo as you’re typing in the terminal. You’re going to type in nano/vi/emacs and edit the file directly. This works until there are > 3-4 files to change. If you’re lucky and all the files are similar then you can create a patch and apply the patch over the computers you care to change. What happens if the files aren’t the same? What happens when all (hundreds) of files are very different and you only want to change a small part of all of them? Using an ASTs is pretty much the only way to do it. This part wasn’t in my proposal for this summer, but if I get the time I’m going to try to get a list of all the computers on a local network, load a requested file from a filtered subset of the computers on the LAN into ASTs for local viewing, comparing and editing. This will let administrators edit entire labs without worrying about differences in comments, order of other values, or white space. If I don’t have time this summer then it’ll make a good project for somebody next summer.

If you’ve gotten this far, thanks for reading. If you have any suggestions for discovering Linux computers on a LAN and editing config files that are owned by root on all of them from a single other computer, please leave a comment. Ideas regarding filtering computers into useful groups on a LAN are also very welcome.

You may have also noticed I said “Ventoo can do <something>”, I’ve actually made some progress so the present tense is appropriate. I’ll have a progress report by the end of this month, and maybe even an overlay for testing. The more config files I can get my hands on the better. More details on helping out should be in my next post.