What are your favorite open source tools?I believe every system administrator is used to open source by now. From Apache to Firefox or Linux, everyone uses it at least a little bit.
However, most open source developers are not good in marketing, so I know that there are hundreds of very good tools out there that very few people know.
To fill this gap, share your favorite open source tool that you use in your day-to-day work.
*I will post mine in the comments.
I love PuTTY [1] !
The PuTTY [2] executables and source code are distributed under the MIT licence, which is similar in effect to the BSD licence. (This licence is Open Source certified and complies with the Debian Free Software Guidelines.)
[1] http://www.chiark.greenend.org.uk/~sgtatham/putty/Notepad++ [1] lightweight, has excellent support for different formats, my main text editing tool in windows.
[1] http://notepad-plus.sourceforge.net/uk/site.htmSynergy [1]
Synergy lets you easily share a single mouse and keyboard between multiple computers with different operating systems without special hardware. It's intended for users with multiple computers on their desk since each system uses its own display.
It is also platform independent.
[1] http://synergy2.sourceforge.net/Nobody mentioned screen yet?
UnxUtils:
[1] This is a port of various gnu shell utilities based on msvcrt.dll so it understands native windows paths - i.e. you don't need to map to a /cygdrive path. This is a key advantage over Cygwin if you have to interact with native windows commands or homebrew CL utilities.
Strings: [2] is a very good way to scrounge through files for items of text. Many, many uses.
Flex: [3] Really designed for writing lexical analysers, with a little bodge artistry and a C compiler it can be used as an uber-grep. I don't use it all that often but it can come in surprisingly handy in that role.
Fetchmail [4] and Procmail: [5] Core of my email system for well over a decade, since I had dial-up internet connectivity. If it ain't broke ...
rdesktop: [6] an open source RDP (terminal services) client that works surprisingly well.
PythonWin: [7], particularly as packaged in Activestate Python [8]. Python on Windows works a lot better than you might think. When used with COM Makepy it's really good for scripting COM APIs.
Wget: [9] an exceedingly useful FTP/HTTP downloading tool.
Leafnode: [10] if you still read any of the newsgroups that still have decent active traffic this is quite a good way to do it. Again, a bit of legacy from my dialup days but it still gets used on occasion.
Abiword [11] and Gnumeric: [12] full featured wordprocessing and spreadsheet software that's far leaner and meaner than OpenOffice.
Xfig: [13] Visio type diagramming tool with an odd user interface. Once you get used to the paradigm it's much easier on my poor old mouse hand than a modern direct maniulation interface. Worth a mention for the ergonomics.
Tcl/Tk: [14] Overshadowed by Perl and Python, Tcl is very easy to embed C code into - it was designed specifically for embedding. Surprisingly useful nonetheless, and the Tk toolkit is very easy to whip up a GUI with. Modern versions support theming so your applications no longer have to look like Motif.
Ghostscript: [15] One of the great unsung heroes of the open-source world. A free postscript interpreter with a whole ecosystem of derived items - PS and PDF viewers, PDF creation tools, printer RIPs and all sorts of Postscript conversion tools. Perhaps most widely used outside open-source circles (if not actively credited) in its role in the back-end of PDFCreator [16]
That's just a sampling of the obscure stuff without mentioning Vim, LaTeX, Firefox, python, gcc, gtk & qt and the Berkeley TCP stack - to name but a few.
[1] http://unxutils.sourceforge.net/My favourite open source tool is rsync.
I use it almost every day and it is still not as famous as it should be:-)
No one mentioned git.
It is not as well known as cvs or svn but I think it will be one day.
7-zip [1]--a file archiver with the high compression ratio. The program supports 7z, ZIP, CAB, RAR, ARJ, LZH, CHM, GZIP, BZIP2, Z, TAR, CPIO, ISO, MSI, WIM, NSIS, RPM and DEB formats.
[1] http://www.7-zip.org/
Wireshark
[1] = Network Protocol analyzer.
Kismet
[2] = A powerful wireless sniffer.
Tcpdump
[3] = The classic sniffer for network monitoring and data acquisition, I use it regularly.
Pound
[4] = The Pound program is a reverse proxy, load balancer and HTTPS front-end for Web server(s).
Trac
[5] = Project management and bug/issue tracking system. Provides an interface to Subversion and an integrated wiki.
Request Tracker
[6] = A free web and email-based bug tracking and trouble ticketing system. Features list, documentation screen shots, and download.
Vmstat
[7] = The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
Iptraf
[8] = The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others.
mc
[9] = Visual shell for Unix-like systems.
Postfixadmin
[10] = Postfix Admin is a web based interface used to manage mailboxes, virtual domains and aliases. It also features support for vacation/out-of-the-office messages.
pwgen
[11] - Automatic Password generation.
Linuxconf
[12] = Linuxconf comes with Mandrake Linux and Red Hat Linux, but is also available for most modern Linux distributions. You've probably encountered this tool before if you use one of these distributions, either as the whole package or in one of its modular components. Multiple interfaces for Linuxconf have been available for years, but now we're up to four: GUI, Web, command-line and ncurses.
Webmin
[13] = Webmin comes with, and was recently acquired by, Caldera Linux. This tool is not only available for most modern Linux distributions, it also runs on most major flavors of UNIX and is available in around twenty languages (though some modules are not available in all of the languages). As you might guess, Webmin is purely a web-based application and a heavily modular one at that.
OpenVPN [14] = SSL/TLS based user-space VPN. Supports Linux, Solaris, OpenBSD, FreeBSD, NetBSD, Mac OS X, and Windows 2000/XP.
[1] http://www.wireshark.org/KeePass [1] is a free open source password manager, which helps you to manage your passwords in a secure way. You can put all your passwords in one database, which is locked with one master key and/or a key file.
[1] http://keepass.info/Screen [1]. It's the most useful tool ever made. Master it and you can be as unto a god, a creature in all places at once.
[1] http://www.gnu.org/software/screen/I use many that I couldn't work without but that I don't consider "not very famous" (openssh, openvnp, apache, rsync, ...). Two very useful little utilities that many may not have heard of sprint to mind:
top with a few useful extra features as well as the prettyBoth can generally be found in standard repositories (they are both in Debian Etch and above) and are relatively painless to compile if your distro doesn't have them.
Edit: another excellent tool that isn't very well known in my experience:
Nagios [1]--Comprehensive IT infrastructure monitoring ensures you can resolve problems before they affect critical business processes....
[1] http://www.nagios.org/dstat [1] - imagine vmstat, iostat, top, ps, as well as apache, mysql, etc. all able to output metrics on the same line at the same interval. cross-referencing app-level metrics with system-level metrics is huge.
siege [2] - better than any other URL hammering tool out there
squid [3] - layer 7 routing and caching, quick and easy
maatkit [4] - MySQL is not the same without it
MySQL Proxy [5] - the example lua scripts are enough to make MySQL snooping painless
[1] http://dag.wieers.com/home-made/dstat/I have been finding that many people don't yet know about Process Hacker [1]. It's on par with Sysinternals' Process Explorer.
Edit in response to Greg's comment:
Sorry for the delay in responding... It also has 2 tabs that show services and TCP/UDP connection info which I think is really nice. You can get the same info in the services tab in Process Eplorer when sorted by tree view, but then you lose the ability sort within the services list.
sudo [1]. I also wrote a similar utility a long time ago (different set of features, lightweight) called Calife [2].
[1] http://www.sudo.ws/sudo/In the security side, I will recommend
Both are well known in the security community, but not very much outside of it.
[1] http://www.ossec.netWinSCP, although I'm not completely sure that's Open Source. If not, it's a toss up between Mailcleaner and HylaFAX.
I have to say Squid. I dont think its all that popular, at least not in the Windows world. We use it for many different things: content filter and port blocker included.
My favorite app is the Window Maker, a very lean and fast Linux window manager (similar to KDE, Gnome, etc).
It is not very famous, but available for most distros (on Ubuntu, do apt-get install wmaker).
Zim Desktop Wiki [1]
Small desktop wiki that works on Linux as well as Windows and OSX.
I use it to keep my tasks organized as well as to document things as I go before putting them into the company wiki.
[1] http://zim-wiki.org/Can you consider a webserver as a tool? If so, nginx [1] has my vote. Else, I would vote for ttcp for throughput testing.
I also find that not a lot of people use xargs. For example, here's a good one I just figured out: how to shred files that have spaces in the filename using find, xargs and shred.
find . -type f -print0 | xargs -0 shred -u -v
I'm a big fan of Filelight [1]. I never knew determining data usage on my hard drive could be so easy or look so pretty.
[1] http://www.methylblue.com/filelight/Sprinkle (http://github.com/crafterm/sprinkle/tree/master) - a software provisioning tool.
It is easier than Puppet [1] or Chef [2], but very powerful. And it is Ruby based, recipes are Ruby scripts.
[1] http://en.wikipedia.org/wiki/Puppet_%28tool%29Ack [1] because it's better than grep:
ack is a tool like grep, aimed at programmers with large trees of heterogeneous source code.
ack is written purely in Perl, and takes advantage of the power of Perl's regular expressions.
I'm also becoming a huge fan of Pandoc [2]:
[1] http://betterthangrep.com/Pandoc is a Haskell library for converting from one markup format to another, and a command-line tool that uses this library. It can read markdown and (subsets of) reStructuredText, HTML, and LaTeX, and it can write markdown, reStructuredText, HTML, LaTeX, ConTeXt, PDF, RTF, DocBook XML, OpenDocument XML, ODT, GNU Texinfo, MediaWiki markup, groff man pages, and S5 HTML slide shows.
I'm afraid that mine are all not very famous and minimalistic, but I live happier since I use them:
Sup [1] - mutt-like console e-mail written in Ruby
wmii [2] - minimalist window manager controlled by a filesystem exported with 9P
Vimperator [3] - firefox plugin to provide a vim-like web brower
MCabber [4] - console mode jabber client
pwsafe [5] - command-line password keeper
AfterGlow [6] - graph-generation from CSV files (for security visualization)
tcptraceroute [7] - traceroute implementation using TCP packets.
pdftk [8] - If PDF is electronic paper, then pdftk is an electronic stapler-remover, hole-punch, binder, secret-decoder-ring, and X-Ray-glasses.
imapfilter [9] - Delete, copy, move, flag, etc. messages residing in mailboxes at the same or different mail servers
[1] http://sup.rubyforge.org/GraphicsMagick [1] - Lesser known alternative to ImageMagick, built as a single executable and offers more regular updates.
mod_evasive [2] - Allows Apache to take evasive action when under brute force or Denial of Service attacks.
WikidPad [3] - Excellent personal wiki for storing just about any kind of information.
NcFTP [4] - Command line ftp client and FTP library (LibNcFTP).
cURL [5] - Lesser known alternative to wget (also available as a library - LibcURL).
PdfCreator [6] - Allows you to print documents to PDF on Windows.
DeVeDe [7] - Lesser known alternative to DVDStyler. I like it better because it gives you greater control over disk layout, and automagically creates a menu from the layout.
InfraRecorder [8] - Open Source GUI for cdrtools on Windows.
KiTTY [9] - Lesser known alternative to PuTTY, can be carried on a USB memory stick.
Strawberry Perl [10] - Lesser known alternative to ActivePerl for Windows, comes with a C compiler (MinGW), and has a portable version.
[1] http://www.graphicsmagick.org/nc: Along with dd it can do wonders in data transfers. We can cloned harddisks/partitions piping nc, gzip and dd together.
dvdisaster: It is really nice tool to create error correction code of CD/DVDs. The ecc occupies much less space then actual disk and even when both ecc and original disk are corrupted (a little) we can still get original data back.
htop: It is way cooler then top as it uses ncurses to show things in nice colors and supports mouse.
tcptrack: It can be used to see live tcp sessions as th4y are getting created and closed. It can sort connections by speed. So you can see where is most of the bandwidth going.
iptraf: It can be used to monitor lot of network interfaces at once to see how much they are being used.
fdupes: It can help in finding duplicate files.
mail-notification: It gives nice pop-up with sound near notification area when new email comes to any of the configured accounts.
kompare: It can give visual diff of two files. You will have to use it once on two similar text files to really understand what I mean by visual diff. My favorite is to compare zone files of primary and secondary DNS to ensure they are consistent.
convert: It is very useful in converting from one image type to another. Specially to eps for Latex documents.
dos2unix / unix2dos: These help in changing files ending with '\r\n' to '\n' and vice-versa. Life saver for cross-platform developers.
indent: If you use vim for coding and do not want to indent some code file by hand. This is really good.
Doxygen: For generating documentation if written in javadoc files for source file in any language.
gftp: This is graphical client for ftp, ssh, http file transfers. It allows resume and asks for conflicting files whether they should be overwritten, resumes, skipped etc.
FileZilla: Similar to gftp above. But it also allows parallel connections to various sites.
Wireshark: I cant say that it is not famous. But it is extremely useful tool that I have used many many times to debug network problems. It is a must for every network administrator.
phpMyAdmin: It can be used to manage MySQL database using web browser from any where very easily. Best part is export options to various formats and easy backup and restore options.
phpPgAdmin: Similar to phpMyAdmin, although not as powerful as phpMyAdmin but it does make life easier.
p7zip: Really nice compression and very helpful in case formats zip / rar are blocked for some reason.
k3b: Really good at burning CD/DVDs. We can burn ISO images or create multi-session data DVDs. We can control what OS support is necessary and whether to use extensions like joilet, rocket ridge etc.
kile: Really nice editor for LaTeX documents. It supports good shortcuts for making dvi, ps, pdf etc and compiling and previewing only selected portion of text
The Apache Foundation's ActiveMQ [1].
I replaced a commercial MQ with it two years ago. It is blazing fast, has lots of HA capabilities and an excellent features list. It is not always easy to configure, but the licensing on the commercial one was starting to look like a pretty decent salary. Time to go open source!
[1] http://en.wikipedia.org/wiki/Apache_ActiveMQPhpMyAdmin [1]--a tool written in PHP intended to handle the administration of MySQL over the Web. Currently it can create and drop databases, create/drop/alter tables, delete/edit/add fields, execute any SQL statement, manage keys on fields.
[1] http://www.phpmyadmin.net/Blat! [1] - A neat little SMTP email sending util - write a script around it to send test emails etc
[1] http://www.blat.net/My favourites that haven't been mentioned yet:
And of course all of the unsung heroes of the *nix command-line world like grep, bash and of course man.
Many of the ones I use have been mentioned but amongst the list of things I always add to a new system:
m4 macro-processor makes adding boilerplate text to config files (and web pages) simple
tidy W3C tool to validate (X)HTML
and, though not in the 'not famous' category in its usual role, I'll risk adding this for its use not as a programmer's tool for software generation but for document and configuration management:
make to simplify maintaining documentation and configs: great with m4 and a good VC system: add an editor of your choice (and not forgetting awk and sed :-)
My favorite sysadmin tools are:
MSYS [1], it's sort-of like Cygwin [2], but designed for building native Windows Apps from Linux style build systems.
[1] http://www.mingw.org/wiki/MSYSAt home, everything is open source, but at work (a Microsoft shop) these are the tools I use on a regular basis...
Performance Analysis of Logs (PAL) Tool [1] - powerful tool that reads in a performance monitor counter log (any known format) and analyzes it using complex, but known thresholds (provided).
PolyMon [2] - monitoring solution that can be used to generate email alerts and analyze historical trends of monitor counters and monitor statuses. It is based on the .NET 2.0 framework and SQL Server 2005.(I like the Powershell support.)
WinSCP [3] - SFTP client and FTP client for Windows.
TortoiseSVN [4] - easy to use Revision control / version control / source control software for Windows. It is based on Subversion. TortoiseSVN provides a nice and easy user interface for Subversion.
WinMerge [5] - differencing and merging tool for Windows. WinMerge can compare both folders and files, presenting differences in a visual text format that is easy to understand and handle.
AxCrypt [6] - file encryption software that integrates seamlessly with Windows to compress, encrypt, decrypt, store, send and work with individual files.
Other favorites already mentioned; Notepad++ [7], PuTTY, TrueCrypt [8], gVim [9], Wget [10], Blat [11], PDFCreator, Wireshark [12], Subversion.
[1] http://www.codeplex.com/Wiki/View.aspx?ProjectName=PALMany tools that I am using on a daily basis were already mentioned. bwm-ng anyone ? It is little live bandwidth monitor.
I'm a huge Puppet fanboy myself, and Systemimager has made my life SIGNIFICANTLY less hateful since I started with it ~2 years ago. iperf is a nifty bandwidth-assessment/troubleshooting tool, and of course the venerable tcpdump!
For me it would be:
poedit [1] - cross-platform gettext catalogs (.po files) editor
ncftp [2] - a very nice ftp client
htop [3] - an interactive process viewer for Linux
nmap [4] - Security Scanner For Network Exploration & Hacking - invaluable detecting all these firewall problems
wireshark [5] - network protocol analyser for Unix and Windows - fantastic tool to trouble shoot networking problems, to learn how protocols work or to recover some passwords, and much, much more
inkscape [6] - a Vector Graphics Editor, that focuses on SVG Compliance
[1] http://www.poedit.net/Chef [1] - a configuration management framework written in Ruby, using a Ruby DSL as a configuration language.
[1] http://wiki.opscode.com/display/chef/HomeFor network monitoring and behavior analysis and forensics I would highly recommend Argus (http://www.qosient.com/argus/).
Anyone mentioned Scapy [1] yet? Very cool packet manipulation in python (and perl, but haven't used ScaPerl).
[1] http://www.secdev.org/projects/scapy/Cacti [1].
It's a pain to set up, but once it's running it's incredibly useful for monitoring anything on your network over time. Unlike tools like Nagios which focus on showing you the status of your systems right now, Cacti is designed specifically for graphing historical data. This is very handy for noticing subtle trends (a slowly filling hard drive, or a gradually warming server room as the AC starts to struggle), or identifying what has changed, and when (this server's load has spiked every day at 2 PM ever since we installed that new software).
It can easily monitor any data that can be collected with SMTP, and with just a bit more work can monitor any data that can be collected by a script at more-or-less-regular intervals.
[1] http://www.cacti.net/
atop
[1] a top like system monitor, but much better.
It makes it easier to spot the bottleneck (hdd, mem, net or cpu).
mtr - My Traceroute, included with most Linux distros but I keep running into folks who haven't heard about it. It's a traceroute/ping mashup, checking every hop of a traceroute continuously and giving you real time stats. You can spot packet-dropping routers and high-latency connections within seconds.
leo [1] - invaluable outliner for data organizing.
[1] http://webpages.charter.net/edreamleo/front.htmlI love CFengine [1] for automatic configuration management on my Debian boxes.
An example on how to edit sysctl.conf [2].
[1] http://www.cfengine.org/Courier [1] - it's a bit of an also-ran in MTAs, but I think it deserves a higher profile. I find it much easier to configure than any of the others and seems to avoid most of the personality quirks of them, too (e.g. qmail's inability to locally relay).
[1] http://courier-mta.org/Nice applications you have stated all, but you definitely need to take a look at this XINE-mplayer [1] and SugarCRM [2] the best collaborative and CRM app
[1] http://www.xine-project.org/homeCrimson Editor [1] is a great source code editor for Windows. It is simple to use, and has syntax highlighting.
[1] http://en.wikipedia.org/wiki/Crimson_Editorscreen - it makes a command line an entire world.
I'd like to mention:
bash Not obvious but (ls -la,ps auwxf,df -h,du -sh,..) are rather useful GPG Thunderbird Dovecot exim perl apache munin RRDtool bind ntpd clamAV SpamAssassin mySQL svn Asterisk pidgin (all the way back to Mark "Asterisk" Spencer) xmpp (ok it is a protocol, but it is open)
collectl
[1] is a very good utility similar to vmstat. But it offers much more options and even supports subsystems like InfiniBand or Lustre. Highly recommended!
iMacros for Firefox addon => automate Firefox via command line
I will try to focus my answer on a tool not so well known (or at least not mentioned).
woof [1] (Web Offer One File), a python script to exchange files using a one-shot minimal HTTP server. No more dependencies thatn python itself. Quite useful if you cannot rely on SSH, SMB or something else.
[1] http://www.home.unix-ag.org/simon/woof.htmlI use Cobbler [1] and Koan a lot. It makes remote installations so much easier.
[1] https://fedorahosted.org/cobbler/lsof - list open files ( ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/FAQ )
Great if you need to know which process has opened a given file !
A nifty little tool that never got any attention but was extremely helpful was retty [1].
If anybody reading this knows assembly and care to do something for this world pleaaaase make retty work with newer systems (It only works on older 32bit systems. And not always...)
[1] http://pasky.or.cz/~pasky/dev/retty/I think Send HTTP tool [1] can be in the list. It is useful to send HTTP request to your URL
[1] http://soft-net.net/SendHTTPTool.aspxI can't see anyone mentioned bc and gnuplot - the poor man's Excel :) I dare to claim these two are one of the best friends of any student in engineering.
My favorites that haven't been mentioned:
and finally
I would have to say Valgrind [1]. As a developer it's extremely useful for realtime debugging of memory problems among others. But it seems no one has yet mentioned it so that means it really isn't that famous in the IT community at large. Maybe just linux/unix developers know about it.
Another one that is also not mentioned but is in the same category is gdb.
No has mentioned gcc either that I can see.
Or gnu make for that matter. I'm sure many of you out there use it all the time but give it no thought.
[1] http://valgrind.org/Webmin - secure LINUX administration tool via a browser. Can do lots of different admin tasks easily, remotely, securely (HTTPS).
I second the PHPMyAdmin to remotely administer MySQL, very powerful and easy.
Image Magick - bulk image processing and command line image manipulation in scripts.
The GNU Windows tool set, lots of gems in here, wget, find, grep, etc. to make Windows more powerful for those of us from *NIX land.
DirSync Pro (Directory Synchronize Pro)
[1]
Directory Synchronize, Synchronization utility.
directorysync.sourceforge.net
jdk1.6 based software that allows you to do bidirectional sync on 2 folders with a gui, and preview of the changes.
[1] http://directorysync.sourceforge.netWorking in the medical industry, it's nice to have a utility like "Eraser" handy. Completely scrubs deleted data from your drive.
Bash - I use it every day. Not just for simple stuff, but being able to chain commands together and script the computer gives me so much power.
My favorite tools on a Linux system are telnet and ping. I am not certain if they are OS or not, but still really useful, also netcat in a similar fashion for net system checking/trouble shooting.
MindTouch. It's developers are actually picked up a lot of steam and thus stands apart from the generalization that "most open source developers are not good in marketing."
Big things happenin'! www.mindtouch.com/blog
I like PgBouncer [1]. It's a connection pooling daemon for Postgresql.
[1] http://pgfoundry.org/projects/pgbouncer/In addition to the numerous good answers already submitted, I'd like to add:
ccrypt [1]: an implementation of AES (Rijndael) for file encryption/decryption
SAGE [2]: a mathematical software suite (which bundles the following two packages)
MPFR [3]: a C library for arbitrary-precision computation
PARI/GP [4]: a C library and command-line calculator for number theory and related applications
GnuPG [5]: e-mail encryption/decryption/authentication software (regrettably, too little used)
[1] http://ccrypt.sourceforge.net/I like the terminal tools
Yakuake .. its a Quake-style terminal emulator based on KDE Konsole technology.
Terminator .. Its a terminal emulator with many cool features like
auto safe session, screen split,find function and many many more.
http://en.wikipedia.org/wiki/Terminator_(terminal_emulator)
enjoy it :)
To complete the list I like to add monit [1]: lightweight and easy to configure monitoring and automated error recovery daemon. Less complex than Nagios and bb4, but far more easy to configure.
config for sshd: (copied from monit website)
check process sshd with pidfile /var/run/sshd.pid
start program "/etc/init.d/sshd start"
stop program "/etc/init.d/sshd stop"
if failed port 22 protocol ssh then restart
if 5 restarts within 5 cycles then timeout
For a complete and useful browser, I use Opera.
SPICEWORKS!!!!
