Do you have adblock enabled?
 
If you can read this, either the style sheet didn't load or you have an older browser that doesn't support style sheets. Try clearing your browser cache and refreshing the page.

(Mercury News)   The earnings numbers stunk and left investors in a funk from playing Jenga and Kerplunk with stocks like Infoblox and Splunk   (mercurynews.com ) divider line
    More: Fail  
•       •       •

417 clicks; posted to Business » on 31 May 2014 at 5:52 PM (2 years ago)   |   Favorite    |   share:  Share on Twitter share via Email Share on Facebook   more»



5 Comments     (+0 »)
 
View Voting Results: Smartest and Funniest
 
2014-05-31 05:21:22 PM  
Splunk is useful for troubleshooting production issues by allowing server-crushing complex log queries without affecting production servers, but its licensing model based on data size rather than seats gets real expensive very quickly.
 
2014-05-31 05:41:13 PM  

vossiewulf: Splunk is useful for troubleshooting production issues by allowing server-crushing complex log queries without affecting production servers, but its licensing model based on data size rather than seats gets real expensive very quickly.


Real expensive, but once they have you by the balls, it's all over.  Our Security team won't let us drop it because they are constantly running queries against data up to a month old, and our sysadmins won't drop it because they use the alert feature to alert on errors that don't affect production but could if ignored long enough.  Everyone uses it if they're lazy.  Our "shadow" networking team, which only handles the enterprise firewalls, has failed for so many years to document how the firewalls are configured that they have to search our Splunk logs to find out if one of their devices is blocking traffic.

But fark, once you start sending any and all logging data from 800+ servers and devices to it, it gets crazy expensive, and unwieldy, too.  Our storage team spent months engineering an agreeable solution for everybody because we didn't have enough space to keep everything online at once.
 
2014-05-31 06:01:08 PM  

Lsherm: vossiewulf: Splunk is useful for troubleshooting production issues by allowing server-crushing complex log queries without affecting production servers, but its licensing model based on data size rather than seats gets real expensive very quickly.

Real expensive, but once they have you by the balls, it's all over.  Our Security team won't let us drop it because they are constantly running queries against data up to a month old, and our sysadmins won't drop it because they use the alert feature to alert on errors that don't affect production but could if ignored long enough.  Everyone uses it if they're lazy.  Our "shadow" networking team, which only handles the enterprise firewalls, has failed for so many years to document how the firewalls are configured that they have to search our Splunk logs to find out if one of their devices is blocking traffic.

But fark, once you start sending any and all logging data from 800+ servers and devices to it, it gets crazy expensive, and unwieldy, too.  Our storage team spent months engineering an agreeable solution for everybody because we didn't have enough space to keep everything online at once.


Very similar situation here. And their email alerts have a nearly impossible to practically improve default ui that makes baby jesus cry. We gave up and started exporting for display in Nagios.
 
2014-05-31 06:48:24 PM  
Yup, I love Splunk and have developed a few apps for it and it's very very powerful
 
2014-05-31 07:27:42 PM  

vossiewulf: We gave up and started exporting for display in Nagios.


Yeah, we're running that, too, but mainly for real time service alerts/problems.  The sysadmins really began going nuts back in the day on Splunk when they were still supporting a huge Blackberry Enterprise Server environment.  BES's would send out error level events for non-service impacting problems, warnings for things that may or may not affect service, and information level events for anything and everything that affected mail delivery, but in a completely nonsensical and useless format.  So you'd get:

Event ID: 15031 | Type: Error | Source: Blackberry Synchronization Agent | Description: BESSync.exe restarted process  <--- This is not a problem

Event ID: 20301 | Type: Warning | Source: Blackberry Messaging Agent | Description: {exam­p­le[nospam-﹫-backwards]e­lpmaxe*com} Unable to save configuration settings or statistics <---- This is not a problem

Event ID: 20154 | Type: Information | Source: Blackberry Enterprise Server | Description: The description for Event ID ( 20154 ) in Source ( BlackBerry Messaging Agent BES Agent 1 ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event:  User <Bob Roberts> not started. <--- This means your CIO isn't getting mail

Before they retired the BES's, the admins had logic trees out the wazoo to create Splunk alerts from random bits of information they pulled from the BES's, all to make sure the important people were getting their mail.  It took them the better part of a year to perfect it, but they did manage to get to the point where they could say with almost 100% accuracy that a message had been delivered.  It really helped stop the stupid panics from our CIO.  He got a ton of email a day and he would routinely miss messages and then say they never showed up.

CIO:  I didn't get this message!
ME:  Steve says you got it.
CIO:  I don't have it!
ME:  Steve says it came in at 2:46 pm.
CIO:  I don't see -
ME:  It's the ninth message you got at 2:46 pm.
CIO:  Oh.  OK, I got it.

A few rounds of that and he started looking a little harder for messages.  It's not like the client team didn't teach him how to search on his blackberry.
 
Displayed 5 of 5 comments

View Voting Results: Smartest and Funniest

This thread is archived, and closed to new comments.

Continue Farking
Submit a Link »
On Twitter






In Other Media


  1. Links are submitted by members of the Fark community.

  2. When community members submit a link, they also write a custom headline for the story.

  3. Other Farkers comment on the links. This is the number of comments. Click here to read them.

  4. Click here to submit a link.

Report