Regional Internet Registries: Quick links

Handy list of links, storing here for future reference when doing research.

A regional Internet registry (RIR) is an organization that manages the allocation and registration of Internet number resources within a particular region of the world. Internet number resources include IP addresses and autonomous system (AS) numbers. Map of Regional Internet Registries. More background here.

– API:
– Request full data here:
– Become a mirror:
– Stats:

– API:
– Downloadable version (redacted):
– Request full data here:
– Become a mirror:
– WHOIS service code:
– Stats:

– API:
– Request full data here:
– Stats:

– API:
– Request full data here:
– Stats:

– API:
– Request full data here:
– Stats:

Posted in Uncategorized | Leave a comment

Bulk elasticsearch file split

Recently needed to split a pretty massive (2M line) file to bulk load into elasticsearch. I’m sure others have run into this same problem. Here’s the code:



filename = "#{ARGV[0]}"
max_commands = 10000  # Max number of commands per file
lines_per_command = 2 # Each elasticsearch command is 2 lines

# Set up to iterate (don't change these)
count = 0
iteration = 0
outstring = ""

# Take the lines of the file and add them to temp string do |lines|
  count = count+1
  outstring << lines.join

  # once we hit our max, write it out
  if count> max_commands
    outstring << "\n"
    path = filename + ".#{iteration}"
    puts "Writing to #{path}", "w").puts outstring
    iteration = iteration+1

    # ... and reset for the next iteration
    count = 0
    outstring = ""


You can all call this: ruby ./es_split.rb [file_to_split.bulk]

Since the file was actively being written to, i then needed to continually send the information to elasticsearch. To do this, i used the following script:


while true; do
echo "[x] Splitting file"
ruby ./es_split.rb $FILE
echo "[x] Sending files to Elasticsearch"
for x in `ls $FILE.*`; do
echo "[x] File: $x"
curl -s -XPOST https://$USER:$PASS@$HOST/_bulk --data-binary @$x;

sleep 10

rm *.$FILE.*
sleep 90


Find the latest version of the code in the intrigue-core repo.


Posted in Uncategorized | 5 Comments

Getting started in information security…. notes to a student

Recently had a college student reach out for advice, and thought i’d share with the class:

My biggest issue with my current education is the broad scale and lack of clear direction on how to achieve my goals. I know that I am very interested in penetration testing. Ethical hacking in general is a very big interest of mine. But as for what area of security, I’m not even sure what the options are.

Cool – sounds like the biggest thing is to explore your options, and decide on a direction knowing full well this may change as you learn more. You may want to try writing your current goals down, and working toward them (or, better yet, backward from the end result).

Penetration testing is still a very… tradecraft career. The best thing you can do is dive in and start learning the underlying systems you’ll be testing. If i had to choose a tester that had a bunch of certs vs a tester that knew (and had admin’d) systems he’d be testing, i’d choose the latter.

Penetration testing has split into some broad specializations – though it’d be best to sample amongst them

  • Mobile
  • Web
  • Network
  • Embedded

Owasp is good for learning web and mobile.

Carnalownage, Metasploit, Offensive security are good for learning network.

Re: certifications – there are some really really good courses certifications – PWK/OSCP, and some really bad ones CEH/CPT,LPT. 

You’ll want to check out netsec’s career thread – this happens quarterly. This will give you a great sample of existing careers, and you can start to research on the things you’ll need to learn.

There are lots of [other] threads on the net about how to get started in infosec.

Here’s another one i wrote a few (zomg, 6) years back, specific to penetration testing

Reddit’s /r/netsec is a great resource for staying on top of what’s happening in the technical security field

Stack Exchange is another one with a bit more of a question/answer focus – good for researching when getting started.

As far as building a reputation while you’re in school, the best things you can do:

  • Get on twitter and start contributing – there’s a strong contingent of security folks on twitter
  • Jump on Github and start publishing tools / code
  • Jump on Bugcrowd and start hacking, building a profile 🙂
  • Publish papers, blogs, code, anything that you can point to as a resume builder
  • Go to conferences, meet folks. Find positive folks that will help you, and find ways to help them.
  • Learn everything you possibly can.

You’re looking for a job in a field that has -10% unemployment, so you’re in the right place at the right time. But be warned, it’s a fast-moving field and requires you to be very motivated if you want to be good.

It’s worth noting that the penetration testing / consultant career path generally requires a significant amount of travel, and can be disruptive to a family lifestyle. This isn’t always true, and there are certainly ways to make it work, but worth thinking about. Thoughts on work / life balance are for another post.

The one piece of advice i give everyone interested in getting into the field: Provide value without asking for anything in return. If you find someone you want to work with, just ask… how can i help? … Guaranteed, they don’t get asked that enough.

Hope this helps.

Also: I ran across this while writing, and there’s some classics here :).

Posted in Uncategorized | 5 Comments

A Logging IRC Bot with Cinch

Yesterday, a friend reached out looking for ideas about logging IRC chats. I pointed him at Cinch, then remembered i’d written something like this in the past. Here’s a complete logging bot that also happens to rotate logs at midnight (handy). It’s largely based on Quintus’s logging plugin, with some tweaks to simplify the configuration.

To use this bot, simply:

  1. Save the bot into a file ‘bot.rb’
  2. Configure the c.nick, c.server, properties in the file.
  3. Install ruby 1.9.1+ and rubygems.
  4. Run the command ‘gem install cinch’ – which will get you the awesome Cinch framework.
  5. Run the command ‘ruby ./bot.rb’ – which will start the bot.
require 'cinch'

# == Logging Plugin Authors
# Marvin Gülker (Quintus)
# Jonathan Cran (jcran)
# == License
# A logging plugin for Cinch.
# Copyright © 2012 Marvin Gülker
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as 
# published by the Free Software Foundation, either version 3 of 
# the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# GNU Lesser General Public License for more details.
# You should have received a copy of the GNU Lesser General Public 
# License along with this program.  If not, see 
# <>.

class Logger
  include Cinch::Plugin

  listen_to :connect,    :method => :setup
  listen_to :disconnect, :method => :cleanup
  listen_to :channel,    :method => :log_public_message
  timer 60,              :method => :check_midnight

  def initialize(*args)
    @short_format = "%Y-%m-%d"
    @long_format = "%Y-%m-%d %H:%M:%S"
    @filename = "log-#{}.log"
    @logfile          =,"w")
    @midnight_message =  "=== The dawn of a new day: #{@short_format} ==="
    @last_time_check  =

  def setup(*)
    bot.debug("Opened message logfile at #{@filename}")

  def cleanup(*)
    bot.debug("Closed message logfile at #{@filename}.")

  ### Called every X seconds to see if we need to rotate the log
  def check_midnight
    time =
    if !=
      @filename = "log-#{}.log"
      @logfile =,"w")
    @last_time_check = time

  ### Logs a message!
  def log_public_message(msg)
    time =
    @logfile.puts(sprintf( "<%{time}> %{nick}: %{msg}",
                                :time => time,
                                :nick =>,
                                :msg  => msg.message))


### It's a bot!
bot = do

  # This block allows us to set up the bot.
  configure do |c|
    c.nick = "just-another-bot"
    c.server = ""
    c.channels = ["#some-channel"]
    c.plugins.plugins = [Logger]
  end # End Configure


Posted in Uncategorized | Leave a comment

OSINT in 2013

Here’s a presentation entitled “OSINT in 2013” I put together for @Isecpartners open forum this week. It’s a work in progress, but gives some sense of where things are at with Tapir. Check it out!

Posted in Uncategorized | Tagged , , | 2 Comments

Google Spreadsheet Bubble Chart for Pentest Results

Google Spreadsheet Bubble Chart for Pentest Results

A bubble chart can make a handy display for pentest results. This is a simple way to identify and display high-impact, low-cost issues.

Posted in Uncategorized | Tagged , , , , | Leave a comment

“You, and Your Research”

I was recently pointed to a great speech entitled “You, and Your Research” given by Richard Hamming of Bell Labs fame. It’s essentially Hamming giving his insight on how to do great work as a scientist. I think it’s relevant for anyone doing infosec research today. Here is my summary, and takeaways. You should read it.

  • Work on important problems.
  • Luck favors the prepared mind.
  • Courage is a characteristic of the successful.
  • Plant acorns to grow oak trees.
  • Follow the greats in your field.
  • Every defect can be looked at as an asset.
  • Knowledge and productivity are like compound interest.
  • Keep track of flaws in your theories.
  • Work on problems you’re committed to.
  • Get emotionally involved, otherwise your subconscious goofs off.
  • Reach out to people outside of your field.
  • Pursue opportunity when its presented.
  • Find and know the important problems in your field.
  • Practice makes perfect.
  • Schedule some dedicated time to make “great thoughts time”
  • Open doors -> more input -> finding the right problems.
  • Zoom out to see the larger problem.
  • You want others to stand on the shoulders of your work.
  • It’s not sufficient to do a job, you have to sell it.
  • Write clearly and well so that people will read it.
  • Learn to give formal talks.
  • Learn to give informal talks.
  • When giving a talk, start slowly and paint a general picture of why its imporant, and give a sketch of what was done.
  • Educate your boss, get other people to ask for what you need.
  • Take advantage of the systems around you to scale yourself.
  • Know thyself & watch thy ego.
  • The appearance of conforming gets you a long way.
  • Don’t spend effort needlessly fighting the system and don’t fool yourself by creating alibis for disappointment.
  • A little extra effort goes a long way with people.

Words to live by.

Posted in Uncategorized | Leave a comment

Hello world!

Welp, we’re back on WordPress after the untimely demise twitterification of Posterous. Welcome. 🙂

Posted in Uncategorized | Leave a comment