netflow on Ubuntu 8.04

To generate and collect netflow data on your Ubuntu 8.04 box, I had to install a couple of packages:

sudo apt-get install fprobe
sudo apt-get install nfdump

fprobe is the daemon that will listen to the traffic and generate a netflow stream to a collector. The nfdump package includes nfcapd, which listens to the netflow stream and generates the flow files on your disk or wherever.

On Ubuntu 8.04 x86-64, fprobe installed an /etc/default file and an /etc/init.d/fprobe file. However, with a quick customization of the files, I found that my changes were ignored and fprobe did not actually run. For now I just ran it manually as root:

fprobe -p -i eth1 -fip localhost:555

The -p flag tells it not to bother putting the interface in promiscuous mode. I did that because I had already put the interface into promiscuous mode manually, after seeing that in /var/log/messages eth0 had been put into promiscuous mode, even though it had no ip address, and was not being used.

The -i flag tells it which interface to get traffic from.

The -f flag tells fprobe to get ip traffic, and is currently required to get any traffic collected at all.

Finally the hostname and port to send the netflow stream to be collected are the last arguments. With that, I had flows being exported!

Next I needed to collect the flows. I was able to edit the /etc/init.d/nfdump file to customize nfdump to work with my machine. You can get it at marmot. The main ingredients are the arguments you feed to nfcapd, here are mine:
DAEMON_ARGS="-D -p 555 -4 -l /home/lincolnr/incoming/nfcap/"

That tells nfcapd to run as a daemon, listen on port 555 for netflow data, use IPv4, and use /home/lincolnr/incoming/nfcap as the storage area for the flow files. This might not be a good production setup, as there is no filtering of where the netflow data comes from. So any netflow stream that is received will get processed, and maybe somebody could do something bad with that somehow...

Then I restarted it with sudo /etc/init.d/nfdump restart.

So running ls -al /home/lincolnr/incoming/nfcap, I started getting some non-empty flow files :)
drwxr-xr-x 21 lincolnr root 16384 2010-06-08 09:26 ..
-rw-r--r-- 1 root root 276 2010-06-08 09:37 nfcapd.201006080932
-rw-r--r-- 1 root root 276 2010-06-08 09:43 nfcapd.201006080937
-rw-r--r-- 1 root root 276 2010-06-08 09:48 nfcapd.201006080942
-rw-r--r-- 1 root root 1120 2010-06-08 09:56 nfcapd.201006080951
-rw-r--r-- 1 root root 756 2010-06-08 10:01 nfcapd.201006080956
-rw-r--r-- 1 root root 1900 2010-06-08 10:06 nfcapd.201006081001
-rw-r--r-- 1 root root 1484 2010-06-08 10:11 nfcapd.201006081006

Yay! Notice that root owns the files and they are world readable. I don't really like that part but it's fine for what I'm trying to do now. Using nfdump, then I can get reports on the traffic:
Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows
2010-06-08 10:20:10.788 0.000 IGMP 192.168.2.1:0 -> 239.255.255.250:0 1 32 1
2010-06-08 10:20:11.497 0.000 IGMP 192.168.2.2:0 -> 224.0.0.251:0 1 32 1
2010-06-08 10:21:14.270 0.000 TCP 208.94.232.254:80 -> 192.168.2.2:54213 1 52 1
2010-06-08 10:21:14.270 0.000 TCP 192.168.2.2:54213 -> 208.94.232.254:80 1 52 1
2010-06-08 10:17:16.609 310.298 TCP 199.18.249.12:59457 -> 192.168.2.2:22 45 2548 1
2010-06-08 10:17:16.568 310.303 TCP 192.168.2.2:22 -> 199.18.249.12:59457 63 58444 1
2010-06-08 10:17:49.883 244.052 UDP 192.168.2.1:1900 -> 239.255.255.250:1900 85 29625 1
2010-06-08 10:22:09.351 0.000 IGMP 192.168.2.1:0 -> 224.0.0.1:0 1 28 1
2010-06-08 10:22:09.804 0.000 IGMP 192.168.2.2:0 -> 224.0.0.251:0 1 32 1
2010-06-08 10:22:17.191 0.000 IGMP 192.168.2.1:0 -> 239.255.255.250:0 1 32 1
2010-06-08 10:22:19.042 0.000 IGMP 192.168.2.1:0 -> 224.0.0.2:0 1 32 1
2010-06-08 10:22:46.366 0.000 UDP 192.168.2.2:138 -> 192.168.2.255:138 2 493 1
2010-06-08 10:23:14.321 0.000 TCP 208.94.232.254:80 -> 192.168.2.2:54213 1 52 1
2010-06-08 10:23:14.321 0.000 TCP 192.168.2.2:54213 -> 208.94.232.254:80 1 52 1
2010-06-08 10:24:15.953 0.000 IGMP 192.168.2.1:0 -> 239.255.255.250:0 1 32 1
2010-06-08 10:24:14.373 0.000 IGMP 192.168.2.1:0 -> 224.0.0.1:0 1 28 1
2010-06-08 10:24:21.405 0.000 IGMP 192.168.2.1:0 -> 224.0.0.2:0 1 32 1
2010-06-08 10:24:19.438 0.000 IGMP 192.168.2.2:0 -> 224.0.0.251:0 1 32 1
Summary: total flows: 18, total bytes: 91630, total packets: 209, avg bps: 1725, avg pps: 0, avg bpp: 438
Time window: 2010-06-08 10:17:16 - 2010-06-08 10:24:21
Total flows processed: 18, skipped: 0, Bytes read: 948
Sys: 0.000s flows/second: 0.0 Wall: 0.000s flows/second: 23076.9

Yay! We can generate netflow data from traffic on our Linux box, and also capture it. I will start another post to explain more about what this is, and why I'm doing it. Hope this helps someone, yes it does work if you trick into working :)

cabin fever

can't be zen all the time

The dead of winter in the midwest has an effect on people. I have been fighting it pretty successfully this year. It's been a struggle, because everyone around me has been struggling. Today it finally got me. It made me angry, because it's the people closest to me whose anguish I now feel. All I could do was cry, and use my words to describe my feelings. You can't be zen all the time. Then I went and had some quiet time to myself, went window shopping for books.

The last two days I had a fun project, I found a rim and tire for my Jeep. I have big tires on my Jeep, and I needed to find a big tire for a spare, and a big rim for the tire to fit on. I ended up buying everything from some really cool guys with eastern European accents, at a little independent used tire shop. They like my ride, and they picked on me for having mismatched lug nuts. I spent the rest of the evening sanding and prepping the rim, and painted it a nice flat black. It turned out better than I really was attempting to make it.

autumn in ohio

Yeah so a lot has happened since I got the Eee... I have turned my fiance into a wife on August 22nd, and had a great time in Aruba on our honeymoon. We got some great gifts and it was all a really special experience. The picture is from the back side of Aruba, we were exploring the lava fields.

I am feeling kruddy today, a flu bug or something has wrestled me to the ground. I'm just trying to think happy thoughts and remember how good I felt last week when I was mountain biking! It's funny, I often start feeling blue and melancholy, and then the next day realize "Oh, I'm sick". My humor is usually a good way to tell how healthy I am. Unfortunately I'm usually not self-aware enough to see it coming.

I've been reading some Melodie Beattie again, "Beyond Codependency". The way she communicates really connects with me. I am reading about intimacy. It's funny how in the past I used to think that obsession was the same thing as intimacy. I'm not sure I really know what intimacy is yet, but at least I'm starting to understand what it is not.

not a mac, not a pc, i'm a eee!

Yeah I picked up a Eee PC. For those unaware, the Eee is product of ASUS or Asustek. It's a little computer like a small laptop, some folks call them "netbooks". Mine is a model 700, known as a "2G Surf", mostly white in color except the back of the display which is baby blue.

Some models of the Eee come with Microsoft Windows, but mine did not. It runs the Eee PC 1.0.2 system. The system is a Linux based system, with some parts of it from the Xandros version of Linux, and finally customized by Asus. This allowed Asus to use free software to build their product, and be competitive with Windows based products. It is really cool technology.

The really cool part is that if you are a programmer or systems person, you can actually make changes to the system *any way you want*. The code for all of the programs, including the operating system kernel and even device driver code, are all freely available. You can edit them in any plain old text editor!

But if you aren't really interested in that kind of thing, but rather want to email, surf the internet, watch YouTube videos, or watch a movie from a file share or similar, you can just do that too! It is designed for ease of use. My fiance is not a technical person, but she doesn't have any trouble using the Eee.


I am also very pleased that I snagged mine in brand-new condition for $106.00 on eBay!

It's like the future is here now; a $100 computer that doesn't need any help from Microsoft. It won't be long now before inexpensive computers that are built without Intel or Microsoft can become the norm.

I gotta run but I posted this from the Eee! Maybe I'll post some technical tweaks and tricks soon!

microsoft and nationalism

During this time of nationalism and Americans pulling together, I couldn't help but notice Microsoft's sponsorship choice for endurance racing in 2009. Yes, they are sponsoring the French automaking industry this year.

I couldn't help but notice that Sun Microsystems is sponsoring Ford.

Still no cigarettes, today is two weeks.

Hanging in there

0 Cigarettes today. I read some information at the American Cancer Association on smoking cessation. Apparently the first three days are the worst. It takes that amount of time for the nicotine to leave your system. And so during that time you are physically adjusting to having lower nicotine levels. That's why folks use the gum or other Nicotine Replacement Therapies. It allows your body to slowly adjust to lower nicotine levels while you cope with the psychological aspects of this process. This evening I was dizzy and wierd. But I'm riding it out. I hope tomorrow I don't get the heebie-jeebies too bad. If so the plan is to go for a walk.

If you're trying to quit hang in there. I quit for a year before, and it was mostly for the wrong reasons. This time is now and I'm in a different place now. I've graduated from therapy and I'm in a healthy relationship. Keep on keepin' on.

Why?

So I have been noticing, in a background noise sort of way, that cigarettes have gotten expensive. Last night I went to get some Marlboro Lights and some Diet Coke, and the cigs were $6.09 per pack. That is too stupid to be real. So if I keep doing a pack a day, a little more on Fridays, that's $200.00 per month on cigarettes. I'm not doing it. I am not spending $200 a month on cigarettes.

I'm pissed off because I didn't want to quit. I quit drinking and that was great but I wasn't ready to give up cigarettes. But I would have to be *fucking stupid* to keep smoking at $6.09 a pack. It *is* too stupid to be real, and I'm not doing it. I put the pack in the trash this morning and I have my nicotine gum with me. grr.

I know it's all for the better but I'm fucking pissed off.

Love fest continues - Okular is the Ferrari of pdf viewers

I am using KDE 4.1 on my OpenSUSE laptop. KDE 4.1 is great, really a nice environment. But the best is Okular.

Okular is a new package that uses libpoppler to replace kpdf. Yes, kpdf is left behind, and now we have Okular. Okular is a KDE 4+ pdf viewer. It is awesome. Okular renders *SO* *FAST*. It renders SO FAST. I can read the UNIX-Hater's Handbook, hit page-down for a few seconds, and then let off, and by the time my eyes focus, the damn page is rendered!!! WHoa! xpdf and ghostview are great but getting crufty, evince and kpdf are great, but Okular is the new hot thing and it is truly wonderful.

If you are a postscript/pdf lover but tired of how slow the old pdf viewers are, check it out. Beauty and speed baby. Okular is the Ferrari of pdf viewers.

I love you Barack

I am watching my president speak. He is plainspoken, but has a vocabulary. He can put lots of ideas together and make a point at the end. He is pragmatic and intelligent. He has more morals than Republicans, but he doesn't talk about them all the time in a hypocritical way like Republicans. I love my new President, best Christmas present ever.

Countries learning to make their own future

Well, it's been a while since I wrote.  There are lots of things I could discuss, such as the fact that I am engaged to be married!  But at the moment I feel inspired to write about something else.

I am amazed at how the world is reacting to what the media calls the "global financial crisis", and what I call "American bankers raiding the world's wealth".  There has been a lot of writing about how unreal and absurd this situation is.  One observation that made me think, courtesy of Lewis Black the comedian, is that, if these bankers that have gotten rich by looting the global economy were leaders of nations, the people of those nations would rise up and kill them.  But since they are leading corporations, people expect them to steal.

I have no doubt that he speaks the truth.

But what I am inspired to write about is not anti-American rage, or shooting thieves.  I am inspired by the creativity and independence that I am seeing from the world.  Small powers, poor countries, and some not so poor, rather than seeing this as a tragedy, see an opportunity here to make their own future.  To back out of the game where other countries try to play ball in the same arena as the United States, and instead make up their own game!  As someone who grew up in what in America is considered a very poor area, I think that I can relate to this independence and ethic.

I'm sure others could write about this from many angles, but as a technical person I only see one angle.  Countries are starting their own efforts to produce their own independent software technology platforms, complete with the infrastructure and salaried staff to develop and support it going forward.  In this new future, they will not be held hostage to pay licensing fees to American corporations for software technologies.  In the spirit of the scientific community in the open, pre-patent times, they are joining a global community of technologists and programmers, to solve practical business problems, and further computer software research, and share freely their thoughts, writings, and even blueprints and computer program source code.

Some people may be familiar with what is called the Free Software movement, or the Open Source movement.  And true, these countries are participating in these same circles.  But this doesn't feel the same as those movements.  This is something new, something that is starting now, in 2009.  Maybe I should come up with a name for it before everyone else realizes it's happening.

One facet of this new independence, is that nations are developing their own computer operating systems.  Many of them are building on the GNU/Linux system.  The GNU/Linux system is a UNIX-like computer system, but with modern support for just about all personal computer and server hardware.  There have also been millions of hours of labor put into making it a more comfortable and easy to use system than UNIX used to be.  The result is a system that comes complete with source code, so that programmers can fix or study or modify any part of the system that they see fit.  This system runs on personal computers, laptops, cellular phones, cash registers, internet servers, military equipment, airplane entertainment systems, and on and on.  GNU/Linux is available on the internet at no cost, and is distributed with a license called the GNU General Public License.  The GNU GPL is kind of the opposite of copyright; it ensures that no matter who uses the software, or what companies want to do with it, that they are in fact forced to share what they have done with the rest of the world.  This license has largely made these events that are happening now possible.

What is happening now is that countries are releasing their own nationalized versions of GNU/Linux.  A version such as this is called a "distribution" of Linux.  This in and of itself isn't necessarily new.  SUSE was originally a German Linux distribution, Mandrake a French, Conectiva a Brazilian distribution.  But just in the last few years we have also seen Ututo, an Argentinian distribution, and Redflag, a Chinese distribution, which is being merged with another asian Linux distribution to form Asianux.  In Korea there is Hansoft Linux.  Recently Russia has announced a project to begin their own operating system, the details of which are not forthcoming, but it is reasonable to think that it will probably be Linux based.  Yesterday I learned of Pardus, which is a Turkish distribution.  I'm sure there are many others that I am unaware of.  I believe that Spain has a distribution based on the Debian distribution, which is not associated with any particular country, but is really kind of the distribution that most Linux software is compiled and tested on.

There may not be as much exciting business news as there was ten years ago, when NYSE decided to standardize on Linux, or when Oracle decided to sell a Linux version.  This is more of a situation where governments need to build their own strong infrastructures, without having that infrastructure based on inflexible platforms that are sold to them from a foreign country, in most cases Microsoft being the vendor and the United States being the foreign country.  And these governments are coming to understand they can stand on their own, and build on the benevolent gifts that have been given to the world from the Free Software and Open Source communities.

happiest of holidays

Well may you and yours enjoy your holiday of choice this season. Mine has been wonderful and I am newly engaged :) Here's to second chances and new first times. *clink clink*

God is in Utah

I never wrote about my trip to Utah. God lives there, right on the border with Colorado. Or at least he was there recently.

I don't know what to say about my trip. I lived in a canyon at 5500 feet looking up at cliffs that were at 7000 feet. I hiked a butte that had petroglyphs on the lower cliff. I saw the leftovers from a mountain lion's lunch. I saw anthills made from crystals of every color, little grape-nuts sized crystals. I was stared down by a bighorn sheep, with his yellow, sideways-slitted eyes. I was out of breath up around 10000 feet looking down on Wolf's Creek Fault, the first time I've ever seen a fault line.

I really don't have words. Check out my pictures.

harmony

So Wednesday morning was our first day after electing Obama. It was a crisp fall morning in Ohio, but the sun was warm. I went outside the office to take a phone call and have a cig. When I was finishing up my call, two young black men were walking to class nearby. They were singing in harmony, and they just kind of floated across the field in the morning sun, singing. It was beautiful, and I see the world as a much more beautiful place than I did before Tuesday night.

fall is here, feels like old times

Sunday morning I rolled out of bed, put on some clothes, and went outside into the chilly air. I pulled out my girlfriend's Nissan, jacked it up, and changed the oil. First time I've rolled out of bed and went out to work on a car like that since I lived out in the country, and drove an 18 year old Honda Accord. The car had over 200,000 miles on it, and I had to work on it almost every weekend just to keep it running, keep driving to work all week.

The air was frosty and damp, and where the sun hit you it felt very warm. Good, good, good feeling. Made the coffee taste good. An hour later I was all done and felt like I had accomplished something. Good way to start a day :)

remember fun?

Well, in spite of the Economy, my boss took our crew out for an evening of fun. We drove go karts and spun each other out and rammed each other into walls. We played laser tag in a place called "LASERTRON", it was very '80s. We giggled like fools. When shooting folks with a laser failed, I hugged their ankles and took them down. We played lame video games. We shot hoops and got tickets. We ate bad pizza. I hit my apexes. I found that one of the apexes was in the middle of the track. One of my cohorts that I don't get along with shot me with his phaser and I drove my sprint car inside him in a corner for a fantastic collision. Remember to have fun :)

may you live in interesting times

Well the past month or so, I've fallen victim to the old curse, "May you live in interesting times". Lots of family visits and funerals. My grandpa died when I was one year old, and last week my grandma's second husband passed away. Watching him suffer from Alzheimer's was very painful. I think that I have some fear to deal with there, fear that it will happen to me. Not because of genes because we didn't share any, but just because it is a horrible fate.
But therapy taught me that we are given our life journey, and it is for us to walk. Whether we get Alzheimer's or not is obviously something that we cannot control, so I need to surrender that to a higher power, and let it go.
There is a great peace in that, if we can find it.
I didn't cry until the AmVets were marching by one by one, saluting his body. My uncle gave me a Springfield M-1 cartridge casing from the 21-gun salute, it meant a lot to me. He has always loved guns, and I have just rediscovered them in the last two years. He knows that we have a similarity there, because being a responsible gun owner and target shooter takes certain psychological traits. I am so glad that we can share that.

trying to do too many things with one computer

I ran into some issues this weekend that made me ask myself, "Am I trying to do too many things with one computer?". The disk on my MacBook got full, and somehow using fetchmail and postfix to flush my inbox made bounce messages go out to everybody in my organization from whom I was not able to store mail. Hooray. Then the NetBeans IDE that I use to do mobile java development for my hobby project, talkLock, decided that since the disk was full, it should explode. Unfortunately deleting some files did not fix it. Then I remembered that since I use home folder encryption, I have to log out for my encrypted are to get shrunk. That didn't fix it either, it didn't complain, but any disk write operations zombied out. A reboot didn't fix it either.

So the next logical step would be to uninstall and reinstall the IDE. But I quit using Windows to avoid stupid crap like this, frankly. If I was using vi and gcc it wouldn't care that at one point the disk was full.

Maybe trying to use one computer to do work like logging in via xterms, running heavy java mail clients, watch movies, listen to music, develop hobby projects, browse the web, edit spreadsheets and documents, play games, etcetera is all too much to expect from an end user desktop system.

Or maybe I should expect postfix to react like that (in UNIX land you would want a mail server with a full disk to fail noisily), and the only real problem is that no one at NetBeans bothered to test what happens when your hard drive gets full.

Of course if you have a few computers then you have the issues with synching things up, and of course you cannot bring clipboard items over from one to another while working.

Or maybe I just need more rest :) One of my tools that I learned in counseling is HALT. Whatever you're thinking about when you are Hungry, Angry, Lonely, or Tired, stop thinking, halt. Take care of your hunger, anger, loneliness, or sleepiness, then you will be clear to deal with things.

Regardless I packed up the Mac and wrote this post on my Linux laptop. Maybe tomorrow I'll try and fix Netbeans on the Mac.

But I just did a new fix in talkLock so I want to test it! Hopefully it'll work fine on the Linux machine.

Network Engineer, 7 Years, and so on

Okay, so to wrap up the whole "How to be a Network Engineer" thing, I'll make a short post.

I ended up moving from being a vendor to being a client. I was The IT Guy for a county-wide library. I had routers and switches and servers. Some of the servers were big Suns. Very fun! I learned a lot there and was very free to explore and build things.

After I was divorced and had settled down some, I decided that it was time to get out of the small town that I had lived in and go to the big city. I was already living there, so it made sense to get a job there.

I did a ~1 year stint at a law firm as The IT Guy. Gone was the freedom of the library, instead I was always being bossed around by shallow, mean, bitchy lawyers all day. It was a Catch 22 situation, and I was not psychologically healthy enough to stand up for myself and define boundaries in that environment. I quit. But I had acquired some training on SQL and gotten comfortable writing database queries and getting around on a database server.

While at the law firm, I had started exploring a Big Ten college campus next to my neighborhood, and took a night class in calculus. It was very hard but a very cool environment.

I took a job at a dry cleaner's, and set about going to school full time to study engineering. I had an Associate degree in electronics at the time. A few weeks later I got a student IT job at the university and was all set to pursue my undergrad degree.

After a year, the guys in the networking department that I hung out with had an opening for me, and I moved over there as a student employee. My new job was to set up Linux servers, work on web and automation scripts, and configure Cisco equipment. Big time real world networking experience! It was a joy.

I learned PHP, Perl, Java, and got more experience with C, C++, bash, SQL.

After a year of that, you can see where this is going. I got a new boss who used to work for an ISP, a state-wide ISP funded by education. I was made an offer and became, yes, a Network Engineer. In only 7 years :)

I am still a senior in Electrical Engineering at the university, but I have not taken any classes since I got my position. We'll see what happens.

Now I work on Cisco, Juniper, SMC, Force 10, and lots of networking equipment. We use BSD servers to do our work. It's not quite the fun and free environment the university was, but it's pretty cool. And now I actually control the internet bandwidth for that Big Ten university, and most of the others in the state :)

We've already got that memo. And others.

Well, I was reading The Unix Hater's Handbook again, and I came across a couple of beautiful gems that I felt like sharing.

"This book is about people who are in abusive relationships with Unix,
woven around the threads in the UNIX-HATERS mailing list. These notes
are not always pretty to read. Some are inspired, some are vulgar, some
depressing. Few are hopeful. If you want the other side of the story, go read
a Unix how-to book or some sales brochures.
This book won’t improve your Unix skills. If you are lucky, maybe you
will just stop using Unix entirely."

"Unix haters are everywhere. We are in the universities and the
corporations. Our spies have been at work collecting embarrassing
electronic memoranda. We don’t need the discovery phase of
litigation to find the memo calculating that keeping the gas tank
where it is will save $35 million annually at the cost of just eight
lives. We’ve already got that memo. And others."

Hahaha, I love it :) That's what I love about Unix folks. We all have a certain gallows humor, a dark understanding that even though we love Unix, we also hate it. We know that it's terrible. But it makes so much more sense to us than other computer operating systems. We are trapped, for good or ill.

I challenge you to find Windows experts with these qualities. People who write about Windows write stuff that reads like sales brochures. I don't like reading sales brochures, because we all know they are bullshit.

Yeah, we've already got that memo.