O R G A N I C / F E R T I L I Z E R: 04.06

Apr 27, 2006

sms: reporting is single-threaded

a fellow mvp, brian rogers, started drilling me about sms reporting the other day (probably because all the smart people are hanging out in san diego drinking margaritas and eating fish tacos in Old Town pretending like they're at a conference). the questions started around how to best setup sms reporting for load. hashing back and forth the basics, it was assumed that sms reporting probably taxed sql server the most ... since the only bottleneck was page faults. brian alluded to a possible problem where the sms reporting function executes one request at a time. we started looking into this and discovered this to be the case. reviewing logs, he discovered that this was indeed the case. i did the same test myself and found that to be happening. the reporting point is, in fact, single-threaded. even though the database can handle multiple queries, the reporting point only sends one at a time. if the browser is closed after a query is sent, the query request does not end. in fact, it'll keep running on the server until it has completed or timed out. this means anyone running a massive report can stall anyone else's request. i can only speculate that the single-threaded operation is to ensure that reporting doesn't overwhelm the sms server/database from fulfilling other functions. the problem here is, what if you have a design where reporting is handled from a database with no clients? that implies that regardless of it functioning solely for the purpose of reporting, it is still single-threaded. it seems the only solution is to add additional reporting points that users can connect to and request reports with.

mom: new mom resource goes live...

if you haven't seen it yet, go check out www.momresources.org. pete and justin, who are becoming legendary mom experts, are doing an excellent job of consolidating content directly related to the successful administration of mom. all of their previous blog material and how-to guides appear to be on that site. their downloads section contains some scripting answers to problems you may be trying to solve today. right now... this very moment... :) they've also removed the frustration of finding some of the not-so-easy to locate materials. just wanted to raise some awareness to a rapidly growing resource that i know i'll be frequenting.

Apr 26, 2006

mom: update to upgrading sql 2005

we upgraded our mom warehouse infrastructure to utilize sql 2005 before the official article was released using the three separate articles i had posted earlier. so far, successful. all dts packages execute post-upgrade! big benefit for me? i can use vs2005 to write the reports now. time to move on to upgrading the production databases. anyway... since that time, microsoft has released an official knowledgebase article on the issues/hotfixes regarding upgrading to sql 2005.

Apr 24, 2006

mom: execute a command or batch file...

i was fiddling around with using this to issue a stop/start sequence on a service under a given condition. the stop/start stuff was pretty easy, but the response execution was a little baffling.

i found that mysterious "use windows command interpreter (not recommended)" dialog is required (i didn't put the parenthesis around not recommended. that was microsoft's doing). anyway, that's the first part. i'll get to that in a minute.

so how do you execute commands in a sequence? the answer is, i don't know. i don't know if it's possible. i don't know if it happens if you order it right. if you notice, the dialog doesn't supply an "up/down" button to move the commands around to sequence them. you'd have to create them in order. even then, does it execute this way? dunno.

the easy way around this is to use a double ampersand. so in order to stop and start the dhcp service, for example, you'd issue this command:

net stop dhcpserver && net start dhcpserver.

this will issue a stop control, wait for the command to finish successfully, then execute a start control. you can do this in a command shell and see how it runs.  (more detail on conditional executions here, if you’re interested.)  back to the first part, in order to execute things like this, you need that (not recommended) setting. otherwise, you'll see something like this in the event log:

Microsoft Operations Manager was unable to create a process to run a batch response.

User Command: %windir%\system32\cmd.exe /c
User Arguments: "net stop dhcpserver && net start dhcpserver"
Command executed: C:\WINDOWS\system32\cmd.exe /c "net stop dhcpserver && net start...
Error details: 3:The system cannot find the path specified.

believe me, i tried all kinds of ways to get this to work, including specifying a variable %windir% in the command line itself to call cmd.exe /c, specifying initial directory, putting the item directly on the command line... none of it worked. the interesting question is why the use windows command interpreter is (not recommended). according to this statement...
Using the Windows command interpreter is not recommended as it exposes customers to command line injection vulnerabilities whereby maliciously constructed instrumentation data could cause the execution of arbitrary code. By separating the application name from the parameters passed to it, secure invocation mitigates the command line injection vulnerability.
there is no secure invocation (calling a program or procedure) when you use this method. what am i trying to say? use a batch or scripted response where possible. it's easy to do this for starting/stopping services. i suppose you could consider this an interim approach and definitely not something to use on secured machines or dmz servers.

mom: all http errors rules

pete has added some great stuff as a follow on to my post regarding http error rules. once again, he illustrates with a magnifying glass on getting to the core of where the problems lie. check out excellent blog entry.

Apr 19, 2006

ds: a few of my favorite repadmin commands...

if you're dirt poor and can't afford a monitoring product to watch over replication status in your environment, you can still get "at-a-glance" capability using repadmin. these are some of the commands i use to see what's going on in my environment. mom, you say? well... errr... yes... i... errr. the story is, we have some domain controllers that do not have the mom agent loaded on them. these are domain controllers in an entirely different domain. however, we still replicate with them as a part of our forest so i have to stay vigilant to their health. (in other words, no mom agents on those...) repadmin /replsum * /bysrc /bydest /sort:delta
displays the source and destination domain controllers, along with their delta.
repadmin /showrepl /csv
displays the source and destination pairs in comma-delimited format which is much easier to read than standard output
repadmin /showutvdec /latency
displays the domain controller, usn value, and sorted by time
repadmin /replicate /force
force replication between two domain controllers

my first korean visitor! (about damn time)

i've been anxiously viewing my clustrmap (it's the picture under the google ads) to finally show me that i had a reader visit from my homeland! i've been tracking it since february... and it took until now to finally see that i have had one visitor. (korea is the bunny shaped country attached at the southeast corner of china and west of the islands of japan.) so what does this have to do with technology? hmmm... nothing, actually. i was just so excited about it, i had to post it. :)

Apr 18, 2006

misc: enabling terminal services remotely...

this may be old stuff, but i've been too busy lately to post anything. i felt obligated to post something, anything. :) anyway, with wmic you can enable terminal services remotely in case you had forgotten to do it during a build. from a command prompt, issue this command:
wmic /node:"servername" /user:"domain\username" rdtoggle where servername = "servername" call setallowtsconnections 1
from inside wmic, issue this command:
/node:"servername" /user:"domain\username" rdtoggle where servername = "servername" call setallowtsconnections 1
  • /node: indicates the remote server name
  • /user: indicates who to grant access to
  • setallowtsconnections: indicates to enable terminal services. 1 enables it. 0 disables it.

Apr 6, 2006

john hann is back...

sort of. he's blogging again, which is good. he's a valuable mom resource and a good friend. apparently, very passionate about technology, mom in particular. he alluded in a previous post that he reached high levels of frustration trying to get product improvements to occur. click here to read an email he sent to steve ballmer last year.

Apr 5, 2006

mom: application log provider (snafu) throws error 25222

david (henson) and i were kicking this around a bit, if you happened to have been watching on the msmom mailing list only to come to a ridiculous fix. anyway, this may come to bite you in the ass at some point or cause unnecessary pain so i had to blog it (not to mention i may forget about it as i slip quietly into senility). i think david may have lost some hair from this one. i encouraged him to call pss. when you're setting up an application log provider, you have to specify the below properties.
  • provider name:
  • provider log type: generic single-line log file
  • directory: x:\test\logs
  • format: generic
  • file pattern: crud.log
none of it is out of the ordinary or difficult to interpret. however, you want to pay careful attention to the way you terminate your directory path. if the directory ends with a trailing backslack, mom may generate event 25222. in the above example, a bad directory path would be "x:\test\logs\". since it's not documented anywhere, i'm encouraging him to get his money back. :) the moral of the story is read your documentation and make sure what you're about to ask is not presented. to give you an idea of how sparse the topic material is covered, here's a snippet from the mom 2005 help file...
Application Log Provider Properties: Directory Edit Allows you to specify criteria to define the files that the application log provider processes. The fields are defined as follows: Format Specifies the format of the log files in the specified directory. Directory Specifies the location of the log file. Pattern Displays the file pattern specified in the File Pattern Edit dialog. Click Add to add an application log file pattern. -------------------------------------------------------------------------------- Did you find this information useful? Please send your suggestions and comments about the documentation to momdocs@microsoft.com.
ummm... no?

Apr 4, 2006

mom: monitoring exc script processes

i recently deployed an exc connector (excellent company, by the way, and very pleasant to work with) to help with trap forwarding with the intent of doing two-resolution state at some point in the future. one of the problems i noticed with the connector was that the script processes running on the exc connector terminated without any warning that i could detect (and subsequently alert on). my goal in this case was to get a little more familiar with wmi notification queries. after some fiddling around, i finally got it to work. alright, so how to do this? here goes:
  1. create a wmi event provider.
    • name: exc_script_processes (i named mine this because i'm just clever like that. name yours whatever you want.)
    • namespace: root\cimv2
    • query: select * from __instancedeletionevent within 89 where targetinstance isa 'win32_process' and targetinstance.commandline like '%cscript%mom%'
    • property list:
  2. create an event rule.
    • provider name: exc_script_processes (or the equally clever name you came up with)
    • criteria:
    • schedule: always process data
    • alert: use a helpful description since wmi events are not pulled raw and do not provide much in the way of useful data.
now any time one of the script processes (mom receiver or mom sender) stops running, you'll be notified. the portion targetinstance.commandline like '%cscript%mom%' looks for any processes that have an execution command line that matches the text and wildcards. in this case, the command line looks something like this:
c:\windows\system32\cscript.exe //job:momreceiver "c:\program files\ ..."
since the wmi event is a notification query, it should run with a schedule of 'always process data'. anyway, looking at the rest of the query...
select * from __instancedeletionevent within 89 where targetinstance isa 'win32_process' and targetinstance.commandline like '%cscript%mom%'
i broke out the query into its elements. you're probably used to interpreting wmi queries by now. for example, select * from win32_process would list all the processes that are running on a machine. in this case, however, we're querying for __instancedeletionevent which signifies when instances are ... yes ... deleted. since we're looking to find when the script processes bomb out, those instances ending, would be captured. within 89 is simply a polling interval. you have to use the within clause when a class does not have a corresponding event provider. in the "event" that it doesn't, you'll receive this error:
'within' clause must be used in this query due to lack of event providers
the where clause is stipulating a condition. targetinstance is an object created as a response to an event (the event being a deletion). with isa we're specifying what class targetinstance belongs to, that being win32_process. the last part targetinstance.commandline indicates what to look for on that command line property to consider it a match. check out this article for more information. it was extremely useful...