No writeDebug() output with command-runner full-command --verbose

vs-cpu-vsx_redirect-to-file.txt.ind (51.6 KB)
vs-cpu-vsx_screen_log.txt.ind (42.7 KB)

[EDIT: added correct screen log file]
(note: rename attachments to .txt)

Trying to debug in command-runner full-command mode with --verbose. No output from writeDebug(). I’ve been seeing this issue for a while, but it’s a real problem for me now trying to develop a complicated script. What’s odd is that if I redirect the command output to a file (>), the output looks right.

Script below, screen output and redirected output attached.

#! META
name: chkp-gaia-vs-cpu-vsx
description: Records the CPU usage for virtual systems
type: monitoring
monitoring_interval: 1 minute
requires:
    vendor: "checkpoint"
    os.name: "gaia"
    vsx: "true"
    role-firewall: "true"

#! COMMENTS
vs-cpu-usage:
    why: |
        High CPU could cause traffic to be dropped, and may result in a notable performance issue.
    how: |
        indeni checks CPU usage each of the the VS processes.
    without-indeni: |
        An administrator could log in and manually check CPU usage.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        CPU utilization is not available for virtual systems, except via CLI or SNMP. It is possible to also get this in SmartView Monitor but it is off by default.

#! REMOTE::SSH
ps -e -o pid,comm | grep _dev

#! PARSER::AWK
{
    writeDebug($0)
}

note: this is on ubuntu bash shell.

Also, I just noticed that if I cat the redirected output (after I redirect the output to a file), I see the same symptoms: the debug output is missing. Something odd here: maybe some control characters affecting the shell display??

Hoping someone else can repro and it’s not just my shell…

Thanks for posting here @Hawkeye_Parker, we will look into this. Meanwhile can you share the command runner version that you are using?

Thanks Alon: 6.0.62

Fwiw, this other issue I reported is a much higher priority for me: Simple shell variable expansion failing in REMOTE::SSH command

I will add some more info to that one.

I’m actually seeing debug prints when I’m running on my machine:

13:15:24.888 [Command-Runner-akka.actor.default-dispatcher-5] DEBUG device_10.3.3.38 - Parsing command 'chkp-gaia-vs-cpu-vsx' , output:  4187 fwk0_dev, aggVars size: 0 values: Map()
2018-07-04 13:15:24,888 20914 DEBUG - Parsing command 'chkp-gaia-vs-cpu-vsx' , output:  4187 fwk0_dev, aggVars size: 0 values: Map()
13:15:24.908 [Command-Runner-akka.actor.default-dispatcher-5] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner -  4187 fwk0_dev
2018-07-04 13:15:24,908 20934 DEBUG -  4187 fwk0_dev

I'm running command-runner with our development build on windows bash (WSL). @Alon_Ashkenazi what is the latest release available for the command-runner?

@Eyal_Roth are you seeing the full debug output? Sometimes I see the first line, but not the rest. You should get:

2018-07-02 11:50:52,950 10961 DEBUG -  4994 fwk0_dev
2018-07-02 11:50:52,950 10961 DEBUG - 11537 fwk2_dev
2018-07-02 11:50:52,950 10961 DEBUG - 11609 fwk3_dev
2018-07-02 11:50:52,951 10962 DEBUG - 11662 fwk10_dev
2018-07-02 11:50:52,951 10962 DEBUG - 11668 fwk13_dev
2018-07-02 11:50:52,951 10962 DEBUG - 11669 fwk1_dev
2018-07-02 11:50:52,951 10962 DEBUG - 11671 fwk7_dev
2018-07-02 11:50:52,952 10963 DEBUG - 11672 fwk5_dev
2018-07-02 11:50:52,952 10963 DEBUG - 11713 fwk4_dev
2018-07-02 11:50:52,953 10964 DEBUG - 11845 fwk15_dev
2018-07-02 11:50:52,953 10964 DEBUG - 12201 fwk14_dev
2018-07-02 11:50:52,954 10965 DEBUG - 12246 fwk6_dev
2018-07-02 11:50:52,954 10965 DEBUG - 12249 fwk11_dev
2018-07-02 11:50:52,954 10965 DEBUG - 12285 fwk17_dev
2018-07-02 11:50:52,954 10965 DEBUG - 12755 fwk12_dev
2018-07-02 11:50:52,954 10965 DEBUG - 12911 fwk8_dev
2018-07-02 11:50:52,955 10966 DEBUG - 12914 fwk16_dev
2018-07-02 11:50:52,955 10966 DEBUG - 13038 fwk9_dev

Actual commands I’m using:

hawk@Tug:~/projects/indeni/indeni-knowledge/parsers/src/checkpoint/gaia$ command-runner.sh full-command --verbose --ssh indeni,indenirocks123! vs-cpu-vsx.ind 192.168.194.121

Redirect and see the complete log of the command:

hawk@Tug:~/projects/indeni/indeni-knowledge/parsers/src/checkpoint/gaia$ command-runner.sh full-command --verbose --ssh indeni,indenirocks123! vs-cpu-vsx.ind 192.168.194.121 > ~/tmp/output.log

Yep, I’m getting:

11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner -  4994 fwk0_dev
2018-07-05 11:45:43,867 15088 DEBUG -  4994 fwk0_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11537 fwk2_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11537 fwk2_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11609 fwk3_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11609 fwk3_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11662 fwk10_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11662 fwk10_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11668 fwk13_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11668 fwk13_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11669 fwk1_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11669 fwk1_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11671 fwk7_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11671 fwk7_dev
11:45:43.867 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11672 fwk5_dev
2018-07-05 11:45:43,867 15088 DEBUG - 11672 fwk5_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11713 fwk4_dev
2018-07-05 11:45:43,868 15089 DEBUG - 11713 fwk4_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 11845 fwk15_dev
2018-07-05 11:45:43,868 15089 DEBUG - 11845 fwk15_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12201 fwk14_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12201 fwk14_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12246 fwk6_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12246 fwk6_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12249 fwk11_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12249 fwk11_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12285 fwk17_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12285 fwk17_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12755 fwk12_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12755 fwk12_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12911 fwk8_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12911 fwk8_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 12914 fwk16_dev
2018-07-05 11:45:43,868 15089 DEBUG - 12914 fwk16_dev
11:45:43.868 [Command-Runner-akka.actor.default-dispatcher-17] DEBUG com.indeni.commands.parsing.awk.CollectorAwkRunner - 13038 fwk9_dev
2018-07-05 11:45:43,868 15089 DEBUG - 13038 fwk9_dev

The latest is 6.0.62, we should release a new one. I’ll work with DevOps to release on a regular basis

@Hawkeye_Parker can you use this: https://indeni-public.s3.amazonaws.com/packages/tools/command-runner/command-runner-6.3.2.zip

@Alon_Ashkenazi wonderful – yes, that resolves my issue. Thank you!

Yup, I had no idea there had been updates to this tool – it would be really nice to get some notification when there’s a new stable version.

@Alon_Ashkenazi is 6.3.2 considered stable?

It is a HUGE improvement to the version I was working with. The test output is vastly improved. Can I pass this on to other IKEs?

We are putting process to release the command runner in a regular manner (for every major release), currently we don’t have a process to ensure the command runner quality. It will be very helpful if you can put tougher a list of test that you think that we should do before releasing it to the community

1 Like

@Alon_Ashkenazi Not exactly clear what you’re asking for. Assume you mean some tests for the command-runner itself. Do you mean you’d like me to write up a list (in words) of tests that I think the command-runner should pass? Or…?

@Alon_Ashkenazi revisiting this old thread. Two questions: