An Information Security Place

I had a long talk with a client yesterday regarding IPS.  They were setting up a nice sized extranet infrastructure to serve their clients, and they needed to build some security into the design before they implemented.  They had already thought of a lot of pieces, and now they were looking at putting in IPS.  They were already being courted by one IPS company, but they wanted to know about others and what the strengths and weaknesses were.

So as I started into the discussion, I diverged a bit from the pure technical discussion and talked about the view of the network as a whole.  Basically, I tried to get them to look at the big picture of what they were buying versus just an IPS as a single silo.  What I talked about was how the one IPS they were looking at was an excellent IPS, but I also told them that they really had no big advantage over any of the other big IPS vendors in the market.  If you look at the Gartner chart for IPS, there are about 5-7 vendors in the magic quadrant.  Basically, the product is a commodity, just like anti-virus and other mature products.  Though some boxes have advantages over others, they all really can do the job.  Most are able to protect multiple segments and can handle multi-gig speeds.  Most have a default set of policies that are not very noisy and protect against the big threats.  Most are HA capable.  Most have fail open or fail close options. Etc, etc, etc.  Some people might disagree here, and I understand that.  One IPS might have a feature that another one does not that may fit a certain need.  But I contend that in a general sense, none of the big ones really have a huge advantage.

So in that light, what are the factors you have to consider?  Well, it really comes down to the intangibles.  Let’s look at a few of those:

Is the company diversified in their product line?  In today’s converging security market, that tells us whether the company is likely to be snatched up or simply disappear, depending on product quality and whether there is someone out there who has money and has a whole in their product line. 

Product diversification may also mean that the company is trying to take a look at the network as a whole versus just one piece.  If they have developed or bought different products that compliment each other and are trying to bring them together in a way that gives insight into the network and allow collaboration, then that type of company is likely planning on sticking around for a while.

In this light, also look at management of the product.  Though this is not exactly an intangible, it is still something that many companies don’t think about.  What about the learning curve for you employees?  Do you already have products from this vendor?  If so, does this new technology fit well into that console, thus lessening the time the your employees need to learn it? If a company fits the diversification example above, they might have a problem in this area.  Of course, if they are serious about making it work, they might very well have an EXCELLENT console.  Take a close look.  You also have to consider the talents of your employees with this factor.

Another intangible is support.  How well do they support their product, keeping in mind that the company with one product may be better at this versus the big one with multiple products?

There are probably many other factors to consider here, but the basic point is that when you are looking at a mature, commoditized product (this does not just apply to IPS, obviously), a decision should not be made on technical issues alone.  Look at your business. Look at your risk.  Look at your employees.  Look at the vendor as a whole.  Compare their position in the market to other vendors.  How do they stack up?  Do they seem to have tunnel vision, or are they trying to diversify?  Make sure you don’t let your technical folks make the decision by themselves and then hand you a PO to sign.  They may like the product in the short term, but you have to think long term.  You might piss off the team for a bit, but you can use the decision as a lesson to help mature your staff.

Vet

I took a class a couple of weeks ago on DLP (data leak/loss prevention).  it was specifically the Websense Content Protection Suite (former PortAuthority).  The class was very good because the instructor spoke a lot about how to position the product as well as the technical workings (good stuff for an SE to know).  One question that arose was whether DLP was a security product.  Now I have a very large definition for the term “security product” because I don’t believe that security can be stove piped like it was in the past (even a switch can be a security product because of its role in availability).

But the point of the conversation was this: do you implement DLP for purposes of protecting data from malicious activity, or do you implement DLP for purposes of protecting against inadvertent data leakage?  Basically, are you protecting against the smart bad guy looking for stuff to steal or the dumb good guy who doesn’t know it is a bad idea to send credit cards in plain text?

I was a little mixed on my opinion on this one.  I understand that you have to protect against the biggest risk.  Most companies are going to experience much more inadvertent loss via SSN’s, CC numbers, customer info, etc. going out through email or some such method.  And because of this, it makes sense to position this type of product in such a way that you are most likely to get a sale.  If you go into a medium-sized shop that has a lot of customers but little-to-no intellectual property, then you are better off positioning the product in this way.

However, let’s look at a few other scenarios:

• Client A is a B-2-B company with no CC numbers, a little customer data, and a huge software app that they developed and is bread and butter to them.
• Client B is a publishing firm that has a new book coming out from a best seller and is afraid that someone will try to steal the manuscript before publishing.
• Client C is a law firm that has all its client data in a SQL db and has not setup any encryption tools yet.  They also have an application that builds legal docs for them and holds the data in a flat file.

Here is where I see DLP having problems, at least from what I have seen so far (PLEASE correct me if I am wrong, especially Mogull).  You might consider positioning it in such a way that shows it can protect against data theft rather than something protecting against inadvertent loss.  Then it IS a security product in that sense of the term.  But the problem I have seen thus far from DLP is that unstructured data is very hard to protect.  It is just not as simple as making a hash of the data and looking for that in a signature.  That type of data just changes too much, and the hash would get broken all the time.

Let’s take Client A.  They are trying to protect their application, so they are protecting against their source code getting out.  Source code is very unstructured, so it is the hardest for a DLP solution to protect.  So Joe Programmer gets paid off by a rival company, and he starts shipping out the code.  If he grabs the source code and just starts dumping it, then any good DLP solution will stop the dump.  But what if he starts breaking it in to pieces and puts it out a bit at a time?  With some experimentation, he can figure out how much gets stopped and how much gets through.  It will be time consuming, but he can get it all out without getting stopped.  Of course, you hope someone notices the dump while he is experimenting and goes to see what is going on, but it is still a feasible scenario.

The same is true for Client B.  A book is also a very unstructured document, and the same problems will arise. 

Now look at Client C.  The first part of the problem is a SQL database.  That can be fingerprinted fairly well and prevention can be done very well.  However, the second part of the problem is unstructured data, which leads to the same issue.

The other problem I see is protecting against streaming protocols.  Store-and-forward protocols are very easy to protect against, but protocols like FTP stream data out, so by the time a solution picks up on the data going out, much of it is already gone.  So if it is not some malicious insider but is Joe Hacker who got in and is stealing your stuff, then you will have lost some data and will likely not have anyone to go after to recover losses.

Anyway, these are some thoughts.  I am sure Rich and a few other people have written about this, but I wanted to get those thoughts out that have been on my mind since I started working on this product line.  I DO know that data, being the crown jewels, is what we have to protect.  I also know that many people forget to look at permissions to data as well as where the data resides, which I see as a flaw in the armor many times.  One of the products out there that can help with that in the Active Directory world is Varonis.  Very good stuff.

Also, Accuvant is starting a data security practice, which tells me that we are taking it VERY seriously.

Vet

I know, I know.  I can answer that question with a resounding “NO” and get on with things.  But seriously, what does it take to even approach security nirvana?  I mean really, there are so many people spouting theories about where we need to go to make the Internet secure.  Then there are a bunch of frickin’ criminal scum suckers over in Russia and China and America and wherever doing everything thing they can to keep fifteen steps ahead of us trying to plug the holes.  And then I take a closer look to see if we really are even plugging the holes (selling product sure as hell doesn’t do it). 

Seriously folks, I know the answer to the question.  But how can we keep going down this road if we can’t even approach a state where we don’t have to look over our cyber shoulder every night and day?  What are we fighting for?  Where did the fight turn into a battle for money instead of a battle for security?  I also know we live in a capitalist society.  I AM a capitalist.  Nothing wrong with making a buck.  But I feel like such a cog among a bunch of cogs.  Where the hell is the wheel??  

I know I sound depressed.  And maybe I am a little.  Maybe it is just because it is 12:35AM right now.  But I just feel like so many of us have lost sight of what it takes to make things secure.  Products have a fit in security.  But with so many of us pushing product after product after product and not looking at security overall, where are we getting to?  When did the industry turn into a churn and burn machine?  This feels like a uphill battle, both ways, in the snow.

I know Alan will probably call me a young, naive punk again (OK, he didn’t call me a punk), but sometimes I have to stop and make sure SOME of my ideals are still there.  otherwise I just become a big glob of compromise, picking up the lint and dirt on my way to security hell…

Vet

I was at a client site the other day…  Wait a minute.  I just realized how often I open posts with that line now.  I feel like Snoopy: It was a dark and stormy night!

Anyway, I was visiting a client the other day (yea, that’s better ), and I was accompanied by my sales guy and a sales guy from a vendor with which Accuvant partners.  My sales guy had invited the partner on the call, and then let me know a couple of days ahead of time that this was going on and that I needed to be there because the vendor’s sales guy was not going to have an SE available from his company.  I am fairly familiar with this particular partners products.  I have used them a lot in the past.  But during the meeting, the conversation turned specifically to a particular product line, and it just so happens that I am not as familiar with this product. 

So long story short, I basically had to admit in the meeting that I did not know the product line very well and I would have to do some research.  Now the customer had no issue with that at all, but I could tell that the partner was none to happy. 

Now generally, I could not care less about what partners think of me.  I have been in trouble before with vendors, and I will be in trouble again I am sure.  But in this particular incident, I felt like I had not done enough prep before hand and had done a disservice to the partner.

Anyway, the meeting went forward and turned to more security-centric talk, such as where they should place IPS, etc.  The sales guys got bored for a while because we got to whiteboarding a bit, but it turned out real well, and the customer ended up giving me some kudos because I pointed out some issues he had not considered.  And several times during the technical talk I pointed out products that the vendor had that could help with certain problems.  So me and my sales guy left feeling like the meeting went well, and I am pretty sure the customer felt the same.  But I still am not sure what the vendor’s sales guy thought.

As a pre-sales engineer, I am expected to know product as well as have in depth security knowledge.  Now I know which one I am better at (three guesses), but I realize the reality of these types of situations.  But as a VAR pre-sales engineer, I am expected to know a BUNCH or products.  It can be a little crazy at times.

So really this is just some thoughts on my blog about this.  I don’t know that I have a specific point.  But for some reason it just struck me to write about this.

Vet

OK, I am officially depressed.  Here’s Richard Bejtlich’s impression of the state of security after one day at BlackHat:

My overall impression from the first day of briefings can be summarized in this manner.

• Existing defenses are absolutely ineffective against current attacks. I am struggling to describe the importance of this insight. It does not matter if you are fully patched, “properly configured,” not running Javascript, or adopting any number of other current defensive stratgies if you use a Web browser that renders modern rich content. Almost none of the techniques described in the Black Hat talks relies upon exploiting vulnerable software. Almost all of them abuse inherent functionality for malicious reasons.
• Detecting current attacks in “real time” is increasingly difficult, if not impossible. Even if you assume attacks are not obscured by encryption, recognizing and understanding the variety of Web-based attacks shown at Black Hat is almost a lost cause. There is basically no way for defenders to address the expanse of the attack surface exposed by “rich Internet applications” and frameworks. I realized that the “rich” in “RIA” refers to the money intruders will make by exploiting Web clients.
• The average Web developer and security professional will never be able to counter these attacks. Intruders are so far ahead of the defenders with respect to tools and techniques that it is simply not possible to prevent the attacks I saw at Black Hat. This statement will probably offend many people but it’s time to face the truth. There is no way to get “ahead of the threat” here.

Holy crap.  What in world am I doing then?  I guess making money off some poor, ignorant bastards who have no idea that every effort we are making is totally useless.  Well, I guess in order to maintain my integrity, I should just quit.

Sheesh.

Vet

OK, now that I am settled in my hotel room in Dallas, I have some time to respond to Alan’s post calling me a hooker (like I said to Alan, at least he called me high-priced).  I will also be responding to a comment left by a reader who goes by the name of Shaneo.  You can read that comment here.

The first thing that strikes me about Alan’s and Shaneo’s comments is that they seem to think that selling products is what makes a person bad.  Either that, or they think I was making that implication.  Alan says:

To me Michael sounds a bit like an expensive call girl talking down on a lowly street walker.  At the end of the day they are both working girls, who work hard for the money, but they are what they are.  As long as Michael is putting the food on the table by selling products to customers, whether they be from a line card that Accuvant offers or from a specific vendor, he is selling nevertheless.

Shaneo says it like this:

You make me laugh! A VAR is still always a VAR - a sales engine. If you were an Independent consultants and didn’t sell any product, then I could support some of your statements.

…don’t put yourself so high and mighty above all the rest…When your a part of the food chain.

I seriously do not get why they think that because I sell products that I am a whore.  My point was never that selling a product was a bad thing.  In fact, my point in the original article wasn’t even to attack vendors, though I’ll admit I was harsh on the vendors (not apologizing, just admitting ).  My point was that I, as the trusted adviser to the client, need to make sure that their expectations are managed so that they can make the right decisions.  I made that point by saying that vendor marketing departments often try to make their products look like they can solve all ills and the client often buys what the marketing department says because they WANT to believe it. 

When Alan asked me in a comment what I would do if I worked for a vendor, I told him straight that I would have to look hard at the vendor before I made a decision “because of the situations I would be in that would require me to sell a product that was not a good fit”.  Do I think every vendor will try to sell something even if it is not a good fit?  No.  And I believe Alan when he says, “It is not some sort of pump and dump scheme over here.” But I also know that it is extremely hard for a salesperson (VAR or vendor) to turn down a sale, and it makes it doubly difficult when you are feeling pressure from above.  So the temptation is there to push the product whether it is a good fit or not.

Now where Accuvant comes into play is that we look at the product that the client is asking about, and since we are the trusted adviser in the situation, we have the leeway to tell them the truth.  If we don’t, then we can lose that status.  Not a good idea for a company that leads with services, not product.  And Alan, you asked, “if Accuvant did not have a product that was a good fit, would you send the customer to EnPointe, Cadre, Fishnet or another VAR?”  Actually, yes, I would.  And I can speak for most, if not all, of Accuvant when I say that they would as well.  That may be hard to believe, but I think you know me well enough to know that I ain’t jerking you around.  In fact, we have contracted with competitors before for stuff that we could not do because of lack of resources or whatever (and no, we did not make them wear Accuvant shirts and not tell anyone where they were from).  We have done that because we place our customers first.  If the competitor gets in and steals the business, then obviuosly we weren’t doing our job in the first place, and we deserve to lose the customer.

Alan also says:

Michael here is another example you site.  The vendor who is upset with you for bringing in his competitor in a deal.  Of course he is.  You would be too.  In fact you are upset by it and you even say that your dander was up because the vendor admitted he wanted another reseller in there.  You wouldn’t mind the vendor suggesting another reseller? See the point.

Well Alan, I see the point you are TRYING to make, but you actually miss it.  Read my paragraph again:

 But what really got my dander up was that I knew that the guy had not brought me in to the client.  In fact, the client requested Accuvant (the client and I were old friends - we had worked at another reseller together).  And in the course of the conversation with me, the sales guy got so flustered that he actually admitted that he had suggested another reseller first (a big mistake on his part that essentially killed his argument, no matter what my argument had been).  This was just pure and simple dishonesty, and it irked me tremendously.

Go to the end.  I wasn’t upset because he suggested another reseller.  I was upset because the vendor was dishonest about saying that he had brought me to the deal when he had actually suggested another reseller first.  That is what makes me wary of vendors.  I have seen that kind of dishonesty time and time again, both from the reseller POV and the client POV.

Another Alan quote:

As long as you are getting paid to put products in at the customer, whether you make and sell them or just sell them, you still sell.  As long as you sell, you are as guilty or innocent, moral or immoral as anyone else in the food chain.

This goes back to my original question.  Why does selling make me guilty or innocent or immoral or moral?  That makes no sense.  It is not the act of selling that makes a person bad.  Guilt and immorality come into play when the sales person or the marketing department or whomever makes false statements to make a sale, and that applies to the VAR or the vendor.  And I know plenty of VARs who sell based on the best spiff that month.  But everything I have seen from Accuvant since before I worked here and after I have been here 9 months tells me that we don’t follow that kind of crap.  Have we had people collect on spiffs before?  Hell yes.  But it was not the driver behind the business.  And if you don’t believe we are on the up-and-up, just ask a customer (thanks again, LonerVamp).

Alan again:

First of all Michael assumes that only someone like a VAR would tell the customer that a case study or lab result are “done in pristine situations”. Why would a vendor be disqualified from saying that?

They’re not.  But do they?  It is not in their interest to do so.

Mr. Shimel again:

Then he talks about telling the customer the truth about how long it takes to install the product. Do you think a vendor is going to lie about this?  Especially if the vendor is selling install professional services along with the product.

Because it is often a bait-and-switch.  Alan, I have seen this so many times it is impossible to name them all.  In fact, one of your competitors in the NAC space does this very thing.  In all honesty, I don’t think the sales person is actually lieing.  However, when he says the product installs in 30 mintues (OK, I exaggerated by saying 5 minutes), he is not telling the full truth.  Does the product physically install in place in that amount of time?  Yes.  They are specifically trying to counter Cisco NAC because they have seen the uber-pain people have gone through trying to implement CleanAccess.  But it takes time to determine the business behind the need for the product, create the policies to fit those needs, get the agent installed on all the workstations, etc.  And yes, a security manager or administrator worth his salt will know the intricacies involved and will know that is a shortsighted claim.  But the fact that he says it and uses it in every sales call creates the need for me to manage the customer’s expectations and let them know all of the other details if installing a product like this.

And if you don’t believe that this is a problem, let me tell you that I have had to convince customers numerous times that getting this product (and others whose salespeople make similiar claims) installed is not just plugging in a couple of patch cables and letting rip.

Anyway, in the immortal words of Forrest Gump:

And that’s all I have to say about that.

I’m going to bed.

Vet

Judging by Alan’s comment to my Managing Expectations post, I think he is a little aggravated with me for picking on vendors.  It probably had something to do with this comment:

…the marketing departments of companies typically make it sound like their product can cure world hunger and make you a sandwich at the same time it is keeping your network totally secure (and it does all that in a nice little 1U appliance that takes five minutes to install and configure). 

Or maybe this:

So as the sales person and sales engineer (often the same person), it is imperative that the expectations for a product are managed up front.  If the customer calls you in and says that the brochure for XYZ Security Widget says that it can perform a certain function, you have to be able to explain if the claim is true or not.  You have to make it clear that often case studies are done in pristine situations.  And you also have to clarify that the “setup” of the widget (yes, the one that takes only five minutes) in a network often means that it was simply screwed into a rack, plugged into the network, and assigned an IP address.  There is usually little to no configuration done on the widget, and it is absolutely worthless in this state.  You have to enlighten the naïve customer by telling him that trade rag product reviews are often rigged (it sucks, but it is true).

I was going to respond in the comments, but it got long, so I thought it was worth a post. OK, here goes.

Alan,

To answer your “what would I do working for a vendor” question, I would honestly have to look long and hard at a vendor before I would go to work there.  Not because they are all a bunch of ” lying no-goodnicks”, but because of the situations I would be in that would require me to sell a product that was not a good fit.  I have interviewed a few times with vendors.  One interview stands out because they asked me what I would say to a client if our product was not a good fit.  I said that I would tell the client it was not a good fit, and the interviewer’s jaw almost hit the floor.  He couldn’t believe I would say that.  But how could I not and stay true to my morals?

I know I give vendors a bad rap, but I have a good bit of experience with them on the customer side and reselling side (this is not my first go ’round as a reseller).  And many, if not most, push their product on everyone, no matter if it is a fit or not.  And then they get aggravated at me for telling the customer the real deal.  Since more often than not Accuvant is the trusted adviser at clients, I am not going to listen to grief from the vendor when I step in as a reseller and try to protect my customer.  I just can’t afford to let a client buy something that is not a good fit.  If I do that a couple of times, I am no longer a trusted adviser.

As an example, I spent 30 minutes on the phone with a vendor sales guy a couple of weeks ago on this very thing.  He was griping at me because I was bringing in a competitor of his into an account he thought he had brought me in on.  The reason I was bringing someone else in was because my client has an internal policy that they have to bring in at least three vendors of any one product before they can make a purchase.   I explained that I could not refuse the customer, especially if he was specifically requesting that I do all the work.  Again, if I don’t help my client, then my status as a trusted adviser gets hurt or lost.

But what really got my dander up was that I knew that the guy had not brought me in to the client.  In fact, the client requested Accuvant (the client and I were old friends - we had worked at another reseller together).  And in the course of the conversation with me, the sales guy got so flustered that he actually admitted that he had suggested another reseller first (a big mistake on his part that essentially killed his argument, no matter what my argument had been).  This was just pure and simple dishonesty, and it irked me tremendously.

I am not saying that all vendors are dishonest.  And I know that vendor product sales make up a huge amount of our revenue at Accuvant.  But I would rather not be put in a situation where I have to choose between making my boss angry by not selling the product or convincing the customer that the product is what he needs when I know it is not.  I just don’t know if I can work in the situation.

Having said all of that, I would really love to hear your deeper opinion on this matter.  Obviously you have had a lot of experience working for vendors, and I want to hear your side on this and how you handle this kind of thing, what you teach your sales people, etc.  I have heard that the vendor side of the house is great, so I want to know what the argument from your side is so I can keep from limiting my options for future employment.

Vet

In Dallas today doing a quick dive into the Websense Content Protection Suite . This is the information leakage protection product formerly known as PortAuthority. I looked at these guys a while back when I was a security manager. They get their accuracy by fingerprinting your data. Basically, they crawl your files and databases and match based on that versus just matching on a string that looks like an SSN or a driver’s license number, which can lead to high false positives (they can match on strings as well).

What also interested me today more was their explanation of when you actually have a compliance violation. Let’s say your HR person sends out an SSN via email. Your first inclination is that you have a violation on your hands. But if you send an SSN without a name or other identifiable info that can be tied to that SSN, then you have no violation. And like I said above, matching on strings can lead to false positives, so you can avoid that with this technology.

They can filter http, ftp, smtp, IM, and some others. As soon as I get a more in depth demo, I will talk more about it.

Vet