What had been still missing from the picture for me, is the ability to log, analyze, and visualize all the data that my smart home generated. I use Splunk (data capture and visualization tool) at work so I decided to give it a try at home and it’s worked out great.

Here’s a Splunk dashboard I created for my home, showing current and historical data from multiple data sources: energy meter, contact sensors, switches, weather data feed, Windows event logs, and some custom PowerShell scripts.

My SmartThings-based smart home setup:

• Samsung SmartThings Hub 2nd Gen
• Amazon Echo Devices
• Various ZigBee/Z-Wave devices
  • Samsung SmartThings GP-U999SJVLAAA Door & Window Multipurpose Sensors
  • Samsung SmartThings GP-U999SJVLBAA Motion Sensors
  • Samsung F-OUT-US-2 SmartThings Outlets
  • Other ZigBee/Z-Wave switches, dimmers, and plugs
  • Samsung ST-CEN-MOIS-1/FTR-US-2 Water Leak Sensors
  • Aeotec HEM G2 whole house energy monitor
  • First Alert ZCOMBO 2-in-1 Smoke Detector & Carbon Monoxide Alarm, Z-Wave
• PowerShell scripts to pull data from openweathermap.org & run/log periodic Internet speed tests.
• Splunk Free

Installing Splunk Free Edition

Download and install Splunk. You will start with the Enterprise version which comes with a 60-Day Trial. After that you can switch to the Free edition. Splunk Free allows indexing up to 500 MB of data per day which has been sufficient for my home logging needs. For my setup I installed Splunk on a 14-year old Windows box with a Intel Core2 Quad CPU Q6600 @2.40GHz – Splunk indexing/query performance has been pretty acceptable.

If your install was successful, you should be able to log into Splunk web by navigating to http://localhost:8000 (or replace localhost with your Splunk server hostname).

If you want to monitor other computers, install Splunk Universal Forwarder on each of those computers. I’ll go through how to configure the Universal Forwarders in a future post.

Enable the Splunk HTTP Event Collector (HEC)

The Splunk HTTP Event Collector is how we are going to send SmartThings events to Splunk. Follow the directions here to set it up. For my setup, I unchecked “Enable SSL”. Make sure you create an Authentication Token and note it down – you will need it later.

Send a test event so you can confirm that it’s working.

curl -k  https://localhost:8088/services/collector/event -H "Authorization: Splunk B5A79AAD-D822-46CC-80D1-819F80D7BFB0" -d '{"event": "hello world"}'

Install/Publish the Splunk HTTP Event Logger SmartThings SmartApp

A very cool benefit of using Samsung SmartThings is the ability to create your own SmartApps, or re-use SmartApp code written by others in the community. To send SmartThings events to Splunk, we will use SmartThingsSplunkLogger SmartApp code by Jason Hamilton/Brian Keifer:

• Go to https://graph.api.smartthings.com and log in
• Choose Locations and select your home location
• Click SmartApps, then “+ New SmartApp”
• Switch to the “From Code” tab
• Copy the source code from here and paste into the “From Code” box in your new SmartApp.

• Choose Create to create your SmartApp.
• Choose Publish/For Me to publish your SmartApp.
• See also:
  • community.smartthings.com discussion thread “Best way to collect logs from the hub?”

Enable the Splunk Logger SmartApp

• In your SmartThings mobile app (instructions here are for Android but should be similar for iOS), switch to the Automation tab.
• Choose “+Add a SmartApp”, scroll to the bottom and select My Apps, then Splunk HTTP Event Logger.
  • 10/28/2022: The “Add a SmartApp link has been moved in the latest app. It should now be in the Automations tab > + in the upper right > add routine > Discover tab and scroll to the bottom.

• Tap on each “Log these…” section and select all the devices you want to monitor.
• In the Splunk Server section:
  • Select Local Server
  • Fill in your Splunk Server hostname/IP
  • Leave Use SSL unchecked
  • Leave Splunk Port as the default 8088 (or change if you want)
  • Fill in the Splunk Authentication Token from earlier.
  • For “Assign a name”, type “Splunk Logger”
  • For “Set for specific mode(s), leave everything unchecked (default)

• Click Save at the top right corner.

At this point, if everything is working correctly, you should start seeing some SmartThings events in Splunk. You may need to wait 15 minutes for events to start coming in.

Build Your Splunk Dashboard

Splunk is fairly easy to use/learn so you should be able to learn the basics pretty quickly. If you are new to Splunk, try going through the Splunk Search Tutorial.

To get you started here are a few of the Splunk queries I use for my dashboard panels.

For the Current Power Consumption Radial Gauge, I use “stats latest(value)” to display the latest value from the Aeotec HEM G2 whole house energy monitor. I also limit the Time Range to the last 60 minutes since I am only interested in very recent data.

sourcetype=httpevent name=power deviceId="118c2de8-3256-44a6-96bd-d1547715fc92"
| stats latest(value)

For the Power Draw History panel, I use the timechart command to show a historical chart:

sourcetype=httpevent name=power deviceId="118c2de8-3256-44a6-96bd-d1547715fc92"
| timechart p50(value)

For the “Indoor Temperature” panel, the query uses a “by” cause to chart multiple series, one from each device. All of these devices are either contact sensors or moisture sensors that also have a temperature sensor built in.

sourcetype=httpevent name=temperature 
|  timechart avg(value) by device

Weather panels such as “Outdoor Temperature”, “Outdoor Humidity”, “Wind Speed” and Rain use data from openweathermap.org’s weather REST APIs. I wrote a PowerShell script that runs hourly to pull the data and appends to a log file, which is forwarded to Splunk. I will cover this in a future blog post.

For the “Contact Sensors & Switches” panel:

sourcetype=httpevent (name=contact OR name=switch)
| eval valueInt = case(value = "closed","1", value="open","2", value = "on","2", value="off","1")
|  stats latest(valueInt) by device

“CPU Load” and “Network Bytes per Sec” panels use data from Windows Event Logs.

CPU Load:
sourcetype="Perfmon:CPU Load" | timechart avg(Value) by host
Network Bytes:
sourcetype="Perfmon:Network Interface" (counter="Bytes Sent/sec" OR counter="Bytes Received/sec") | timechart sum(Value) by host

“WAN Speed” panel shows hourly Internet speed test results from a custom PowerShell script.

Hope this helps you to get started and move your smart home to the next level. In future posts, I will write more about installing and configuring Splunk Universal Forwarders, sending other types of data to Splunk, including Windows Events Log, weather data from openweathermap.org, and custom PowerShell scripts to log other data as well as querying Splunk data and send out alerts if certain conditions are met.

Troubleshooting Tips

If SmartThings events are not showing up in Splunk try looking at SmartThings Live Logging logs. Log into SmartThings Groovy IDE then choose Live Logging menu. When a SmartThing event occurs (such as when a door contact sensor is tripped), you should see a log entry written:

See Also

• Sending WAN/Internet Bandwidth Usage Data to Splunk from Tomato Routers using Splunk HEC

The post Monitor & Visualize Your SmartThings Smart Home with Splunk appeared first on Chinh Do.

$dataSource = "MyFeed" # replace this with your datasource name
$gsaUrl = "http://testsearch.example.com:19900/xmlfeed"; # replace this with your GSA URL

$xml = @"
$dataSource incremental
"@
   
Write-Output "Submitting feed to $gsaUrl."

$body = "feedtype=metadata-and-url&datasource=$dataSource&data=$xml"
$contentType = "multipart/form-data"
$length = $body.Length

Write-Output "Posting feed to GSA:"
Write-Output $xml

$r=Invoke-WebRequest -uri $gsaUrl -method POST -body $body -ContentType $contentType -UserAgent "Ps-GsaFeedSubmitter"

See also Feeds Protocol Developer’s Guide

The post Posting a feed to Google Search Appliance (GSA) from PowerShell appeared first on Chinh Do.

function Is-LastWeekDayOfMonth([DateTime] $d) {
    return [DateTime]::DaysInMonth($d.Year, $d.Month) - $d.Day -lt 7
}

Similarly, finding the last specific weekday of the month involves finding the last weekday, and the difference between that weekday and the target weekday:

function Get-LastFridayOfMonth([DateTime] $d) {
    $lastDay = new-object DateTime($d.Year, $d.Month, [DateTime]::DaysInMonth($d.Year, $d.Month))
    $diff = ([int] [DayOfWeek]::Friday) - ([int]
    $lastDay.DayOfWeek)
    if ($diff -ge 0) {
        return $lastDay.AddDays(- (7-$diff))
    }
    else
    {
        return $lastDay.AddDays($diff)
    }
}

The post Finding the Last Friday (or any weekday) of the Month in PowerShell appeared first on Chinh Do.

The “2>&1” at the end allows Powershell to detect errors and bail out if you have $ErrorActionPreference = “Stop”

If you still have problems, download and use EchoArgs.exe to see exactly what PowerShell sees.

Example

function Get-MSWebDeployInstallPath(){
  $path = (get-childitem "HKLM:\SOFTWARE\Microsoft\IIS Extensions\MSDeploy" | Select -last 1).GetValue("InstallPath")
  $path = "${path}msdeploy.exe"

  if (!(Test-Path "$path")) {
    throw "MSDEPLOY.EXE is not installed. See http://go.microsoft.com/?linkid=9278654"
  }

  return $path
}

$msdeploy = Get-MSWebDeployInstallPath

& $msDeploy -verb:sync -source:recycleApp `
  -dest:recycleApp=`"${webSiteName}`"`,recycleMode=StopAppPool`,wmsvc=${server}`,userName="${userName}"`,password="${password}" `
  -allowuntrusted -debug 2>&1

The post Running MSDEPLOY from PowerShell appeared first on Chinh Do.

In my library script file, I have the following functions to play sound files and to speak any text:

function PlayMp3($path) {
    # Use the default player to play. Hide the window.
    $si = new-object System.Diagnostics.ProcessStartInfo
    $si.fileName = $path
    $si.windowStyle = [System.Diagnostics.ProcessWindowStyle]::Hidden
    $process = New-Object System.Diagnostics.Process
    $process.startInfo=$si
    $process.start() 
}

function PlayWav($path) {
    $sound = new-Object System.Media.SoundPlayer;
    $sound.SoundLocation="$path";
    $sound.Play();
}

function Say($msg) {
    $Voice = new-object -com SAPI.SpVoice
    $Voice.Speak($msg, 1 )
}

If you like the text-to-speech feature but find Windows’ speech engine lacking, check out Ivona. It’s a commercial text-to-speech engine but you are allow to generate and download short speech files for free personal use. Now, my script can nicely interrupt me to tell me when it’s done. Other online text-to-speech engines: vozMe, SpokenText.

If Making Noise Is Not Your Thing

If making noise is not your thing, consider displaying a message in the Notification Area. Here’s the code (courtesy Microsoft TechNet):

function Get-ScriptName {
    $MyInvocation.ScriptName
} 

function DisplayNotificationInfo($msg, $title, $type) {
    # $type - "info" or "error"
    if ($type -eq $null) {$type = "info"}
    if ($title -eq $null) {$title = Get-ScriptName}

    [void] [System.Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
    $objNotifyIcon = New-Object System.Windows.Forms.NotifyIcon 
    # Specify your own icon below
    $objNotifyIcon.Icon = "C:CdoScriptsFolder.ico"
    $objNotifyIcon.BalloonTipIcon = "Info"  
    $objNotifyIcon.BalloonTipTitle = $title
    $objNotifyIcon.BalloonTipText = $msg

    $objNotifyIcon.Visible = $True 
    $objNotifyIcon.ShowBalloonTip(10000)
}

The post Give The Power of Speech and Sound to Your PowerShell Scripts appeared first on Chinh Do.

Here’s how:

function InvokeAndCheckStdOut($cmd, $successString, $failString) {
    Write-Host "====> InvokeAndCheckStdOut"
    $fullCmd = "$cmd|Tee-Object -variable result" 

    Invoke-Expression $fullCmd
    $found = $false
    $success = $false
    foreach ($line in $result) {
      if ($line -match $failString) {
       $found = $true
       $success = $false
       break
      }
      else {
       if ($line -match $successString) {
        $found = $true
        $success = $true
        #"[InvokeAndCheckStdOut] FOUND MATCH: $line"
        break
       }
       else {
        #"[InvokeAndCheckStdOut] $line"
       }
      }
    }

    if (! $success) {
      PlayWav "${env:windir}\Media\ding.wav"
      throw "Mvn command failed."
    }

    Write-Host "InvokeAndCheckStdOut <===="
}

function InvokeMvn($cmd) {
    InvokeAndCheckStdOut $cmd "BUILD SUCCESSFUL" "BUILD FAILED"
}


InvokeMvn "mvn clean install"

See Also

• Tee-Object and Invoke-Expression in PowerShell

The post Automating Maven with PowerShell appeared first on Chinh Do.

Invoke-Expression "mvn clean install | Tee –variable result”

The following, which I guess is what most people try first, doesn’t work (at least in PowerShell V1). I guess because you are storing the result of the “Invoke-Expression” command itself into the variable instead of “mvn clean install”.

    Invoke-Expression "mvn clean install” | Tee -variable result

Wrapping Invoke-Expression in parenthesis (see below) works, but has a drawback: the output is not written to Standard Out until the whole command finishes.

    (Invoke-Expression "mvn clean install”) | Tee -variable result

The post Tee-Object and Invoke-Expression in PowerShell appeared first on Chinh Do.

An empty file created immediately when you call this method. Remember to clean it up when are you done!

The post Create a Temporary File in PowerShell appeared first on Chinh Do.

• Thread synchronization: Wait and Pulse demystified. By Nick Butler.
• Use a shortened GUID for file names and request strings. ShortGuid – A shorter and url friendly GUID class in C#. By Dave Transom.
• Top Ten Tips to Using XPath and XPointer. By John E. Simpson.
• Client-side caching for script methods access in ASP.NET AJAX, by Jeffrey Zhao.
• A very nice two-part introduction to jQuery, by Rick Strahl.
• A COMPARISON OF MICROSOFT’S C# PROGRAMMING LANGUAGE TO SUN MICROSYSTEMS’ JAVA PROGRAMMING LANGUAGE. By Dare Obasanjo.
• Silverlight
  • Silverlight 2 End to End Tutorial: Building a Digg Search Client, by Scott Guthrie.
  • Fluid Canvas in Silverlight, by neo.
• WCF
  • Performance Improvement for WCF Client Proxy Creation in .NET 3.5 and Best Practices. By Wenlon Dong.
  • A comprehensive guide to using MsmqIntegrationBinding with MSMQ 3.0 in WCF. By Simon Evans.
  • If your WSE 3.0 Mutual Certificate Authentication web service runs very slowly, try setting establishSecurityContext to true.

PowerShell

• PowerShell Community Extensions has very useful cmdlets such as Get/Set-Clipboard, Write-GZip, Start-Process, Select-Xml and many others.
• John D. Cook’s PowerShell Cookbook has lots of useful PowerShell recipes.

Apps and Tools

• Microsoft Windows SteadyState (free) is great for maintaining public-access computers.

Other Stuff

• Anatomy of a malware scam – The evil genius of XP Antivirus 2008. By Jesper M. Johansson. If you’ve had to help your relatives and friends clean up after XP Antivirus 2008/2009, read this very detailed analysis of the malware.

The post Bag of Links #2 appeared first on Chinh Do.

General Programming

• Some code optimizations can actually slow down your app. Beautiful Code: False Optimizations by sernaferna.
• Quad core + 8GB RAM + Server 2008 with Hyper-V = A Great Dev Environment. By Craig Lussier.
• I started using JQuery a few months ago and it’s now my favorite Javascript library. Combine with Visual Studio 2008’s Javascript intellisense, and you have scripting nirvana.
• jqModal is a modal dialog plug-in for JQuery. Worked great for a project I was working on two months ago.
• Safari Books Online is a great online programming books service. For a monthly fee, you get to read several books using your browser. The browser interface is not great, but the price is much lower than actually buying the book.
• Randomly ordering an array is simple. That’s what I thought until I ran across Techniques for Randomly Reordering an Array (by Scott Michell).

.NET/C# Stuff

• Eugene Osovetsky explained a few peculiarities when consuming WCF services from Visual Studio 2003: Solving the “disappearing data” issue when using Add Web Reference or Wsdl.exe with WCF services.
• If you have a DataReader and don’t intend to read all records from it, call IDbCommand.Cancel before you call IDataReader.Close so that the rest of the unread records are skipped.
• Scott Hanselman: Assembly Fiefdoms: What’s the Right Number of Assemblies/Libraries?
• VS.NET Tip of the Day: Reusing C# Source Code Across Multiple Assemblies. By ShawnVN.
• Setting the log file location at runtime with a DOM configured log4net. By Keyley on Kode.
• WCF – Recommended Settings for Tracing and Message Logging. MSDN.
• WCF – Enabling Performance Counters. By Nicholas Allen.
• One of these days I’ll get back into Windows Mobile programming. Adjust Your Ring Volume For Ambient Noise, Chris Mitchell.
• Did you know ReSharper supports wildcards in its Go To Type dialog box? Joe White’s 31 Days of ReSharper series is a must read for ReSharper fanatics like me.
• Is image processing in C++ a thing of the past? Discover Enhanced Image Manipulation with GDI+ (by Tade Oyebode).
• DataContractSerializer can be used to deserialize WCF messages.
• Unraveling the Mysteries of .NET 2.0 Configuration, by Jon Rista.

Database

• Stefan Delmarco talked about the NEWSEQUENTIALID function in SQL Server 2005.
• Mutexes are fun: Application Locks (or Mutexes) in SQL Server 2005. Mladen Prajdic.

Windows

• Experiencing strange issues with your Windows (like dialog boxes that don’t show up)? You may be having a desktop heap issue. Try these articles:
  • Windows XP flakiness – solved. By Kevin Dente.
  • “Out of Memory” error message appears when you have a large number of programs running. Microsoft Support.
• If IE crashes when you try to open SharePoint documents, try this fix: regsvr32 “C:\Program Files\Microsoft Office\OFFICE11\OWSSUPP.DLL”.

Software, Tools, etc.

• If you have a Linksys WRT54* router, I highly recommend loading Tomato firmware. I have been using it for about 6 months now and it’s so much better than the built-in Linksys firmware. Tomato’s QOS works great to make sure my Vonage phone line remains usable at all times.

PowerShell

• Ben Pierce posted a series of very useful PowerShell command-line demos: Demo1 (Administering Windows), Demo 2 (Administering Servers in bulk), Demo 3 (How do I Know Which Class to Use), Demo 4 (Administering Hyper-V).

Something Different

• World War is a very nicely done digital animation by Vincent Chai. My son can watch it for hours.
• Your browsing history can be used by any web site out there to guess your gender. Using your browser URL history to estimate gender, by Mike on Ads.
• Humans can fly (video).
• A fascinating into the human mind: The Interpreter (Has a remote Amazonian tribe upended our understanding of language?). By John Colapinto.

The post Bag of Links #1 appeared first on Chinh Do.