TL;DR Not able to see syslog logs in influxdb
Environment:
OS: mac os mojave
telegraf version: 1.8
influxdb version: 1.6.4
So I wanted to view logs in chronograf and figured out from the set of input plugins offered in telegraf to use the syslog plugin.
I have followed the instructions here, but have added the steps here too for easy read.
I installed rsyslog via homebrew as follows:
$ brew install rsyslog
Added the following in /usr/local/etc/rsyslog.conf:
$WorkDirectory /tmp/rsyslog # temporary directory for storing data
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
$ModLoad imudp #loads the udp module
#listen for messages on udp localhost:514
$UDPServerAddress localhost
$UDPServerRun 514
*.* ##(o)127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format
restarted rsyslog:
$ sudo brew services restart rsyslog
I configured telegraf as follows:
# # Accepts syslog messages per RFC5425
[[inputs.syslog]]
# ## Specify an ip or hostname with port - eg., tcp://localhost:6514, tcp://10.0.0.1:6514
# ## Protocol, address and port to host the syslog receiver.
# ## If no host is specified, then localhost is used.
# ## If no port is specified, 6514 is used (RFC5425#section-4.1).
server = "tcp://localhost:6514"
and restarted telegraf as follows:
$ brew services restart telegraf
But my expectation was to see syslog measurement inside of telegrafdatabase
I wrote the following python script to log to syslog hoping that it would appear in the database:
import logging
import logging.handlers
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)
handler = logging.handlers.SysLogHandler(address='/var/run/syslog')
my_logger.addHandler(handler)
my_logger.debug('this is debug')
my_logger.critical('this is critical')
but to no avail.
What could be wrong here? If so is there a log file I could check in?
EDIT 1:
So I troubleshooted rsyslog using the rsyslogd -N1 command and found some issues:
The configuration file must be in /etc/rsyslog.conf.
The configuration file working directory i.e; /tmp/rsyslog wasn't found so created one.
There were some errors in rsyslog.conf file (can be seen in edits of this question)
After doing all the above I restarted rsyslog and also influxdb and telegraf and checked again if there were any errors by running the rsyslogd -N1 command and the following is the output:
rsyslogd: version 8.37.0, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.
Still the same issue persists.
EDIT 2:
related post: syslog-plugin-from-remote-server
Related
On running the command vctl config list platform.driver , I get the configuration for fake device agent and my remote agent. It must be because I choose "Yes" for installing fake device on the master driver on running vcfg initially.
Can anyone suggest me how to remove the fake device only keeping the configuration for my modbus device agent??
I tried removing the platform.driver vctl remove platform.driver and reconfiguring my VOLTTRON environment. (this time without installing fake device) Still, I am getting the config listed for both the devices.
You can use the below command to delete a specific configuration by name from the store
volttron-ctl config delete <agent vip identity> <configuration name>
https://volttron.readthedocs.io/en/6.0/core_services/config_store/Commandline-Interface.html
I've done alot of research, and I can't find anything which actually solves my issue.
Since basically no site accepts mitmdumps certificate for https, I want to ignore those hosts. I can access a specific website with "--ignore-hosts (ip)" like normal, but I need to ignore all HTTPS/SSL hosts.
Is there any way I can do this at all?
Thanks alot!
There is a script file called tls_passthrough.py on the mitmproxy GitHub which ignores hosts which has previously failed a handshake due to the user not trusting the new certificate. Although it does not save for other sessions.
What this also means is that the first SSL connection from this perticular host the will always fail. What I suggest you do is write out all the IPs which has failed previously into a text document and ignore all hosts which are in that text file.
tls_passthrough.py
To simply start it, you just add it with the script argument "-s (tls_passthrough.py path)"
Example,
mitmproxy -s tls_passthrough.py
you need a simple addon script to ignore all tls connections.
import mitmproxy
class IgnoreAllTLS:
def __init__(self) -> None:
pass
def tls_clienthello(self, data: mitmproxy.proxy.layers.tls.ClientHelloData):
'''
ignore all tls event
'''
# LOGC("tls hello from "+str(data.context.server)+" ,ignore_connection="+str(data.ignore_connection))
data.ignore_connection = True
addons = [
IgnoreAllTLS()
]
the latest version ( 7.0.4 for now) is not support ignore_connection feature yet,so u need to install the main source version:
git clone https://github.com/mitmproxy/mitmproxy.git
cd mitmproxy
python3 -m venv venv
activate the venv before startup the proxy
source /path/to/mitmproxy/venv/bin/activate
startup mitmproxy
mitmproxy -s ignore_all_tls.py
You can ignore all https/SSL traffic by using a wildcard:
mitmproxy --ignore-hosts '.*'
I have a command service app start-demo requires me to type sudo service app start-demo in the command line.
I used sudo(service app start-demo) and sudo(sudo service app start-demo) but I still get
Warning: sudo() encountered an error (return code 1) while executing 'sudo service app start-demo'
I have no problem executing that as a command line in a terminal.
I am not sure if SADeprecationWarning: counts as a failure to fabric?
Thanks.
user#box:/var/lib/app$ fab kickstart
You are installing prereqs..........
### Install Prereqs for Populate ###
No hosts found. Please specify (single) host string for connection: localhost
[localhost] Login password:
### I am starting demo ###
[localhost] sudo: sudo service app start-demo
[localhost] out: Starting demo
Fatal error: sudo() encountered an error (return code 1) while executing 'sudo service app start-demo'
Aborting.
Disconnecting from localhost... done.
the code
def pserve():
print '### I am starting demo ###'
#with settings(warn_only=True):
sudo('sudo service app start-demo')
#sudo('service app start-demo')
either sudo command will fail.
/etc/sudoers
# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#
Defaults env_reset
# Host alias specification
# User alias specification
# Cmnd alias specification
# User privilege specification
root ALL=(ALL) ALL
# Allow members of group sudo to execute any command after they have
# provided their password
# (Note that later entries override this, so you might need to move
# it further down)
%sudo ALL=(ALL) ALL
#
#includedir /etc/sudoers.d
# Members of the admin group may gain root privileges
%admin ALL=(ALL) NOPASSWD:ALL
This is prolly related to this mention in the faq, but also if the command doesn't return 0 (unix standard for all good) it'll fail-fast unless you tell it to warn only.
I tried this but it shows me the middle finger:
~/Software/android-sdk-linux_x86-1.6_r1/tools$ sudo ./monkeyrunner -v
help.py Exception in thread "main" java.lang.IllegalArgumentException:
Bad level "help.py" at
java.util.logging.Level.parse(Level.java:336) at
com.android.monkeyrunner.MonkeyRunnerOptions.processOptions(MonkeyRunnerOptions.java:130)
at
com.android.monkeyrunner.MonkeyRunnerStarter.main(MonkeyRunnerStarter.java:192)
It is on a freshly installed java, jdk, jre, Eclipse, AndroidSDK, ADT
Any ideas?
Why sudo ?
You shouldn't use it.
The other problem is that you are missing the log level for -v:
Usage: monkeyrunner [options] SCRIPT_FILE
-s MonkeyServer IP Address.
-p MonkeyServer TCP Port.
-v MonkeyServer Logging level (ALL, FINEST, FINER, FINE, CONFIG, INFO, WARNING, SEVERE, OFF)
Also, be sure you have help.py in your current working directory or specify its path.
I've followed the basic CherryPy tutorial (http://www.cherrypy.org/wiki/CherryPyTutorial). One thing not discussed is deployment.
How can I launch a CherryPy app as a daemon and "forget about it"? What happens if the server reboots?
Is there a standard recipe? Maybe something that will create a service script (/etc/init.d/cherrypy...)
Thanks!
Daemonizer can be pretty simple to use:
# this works for cherrypy 3.1.2 on Ubuntu 10.04
from cherrypy.process.plugins import Daemonizer
# before mounting anything
Daemonizer(cherrypy.engine).subscribe()
cherrypy.tree.mount(MyDaemonApp, "/")
cherrypy.engine.start()
cherrypy.engine.block()
There is a decent HOWTO for SysV style here.
To summarize:
Create a file named for your application in /etc/init.d that calls /bin/sh
sudo vim /etc/init.d/MyDaemonApp
#!/bin/sh
echo "Invoking MyDaemonApp";
/path/to/MyDaemonApp
echo "Started MyDaemonApp. Tremble, Ye Mighty."
Make it executable
sudo chmod +x /etc/init.d/MyDaemonApp
Run update-rc.d to create our proper links in the proper runtime dir.
sudo update-rc.d MyDaemonApp defaults 80
sudo /etc/init.d/MyDaemonApp
There is a Daemonizer plugin for CherryPy included by default which is useful for getting it to start but by far the easiest way for simple cases is to use the cherryd script:
> cherryd -h
Usage: cherryd [options]
Options:
-h, --help show this help message and exit
-c CONFIG, --config=CONFIG
specify config file(s)
-d run the server as a daemon
-e ENVIRONMENT, --environment=ENVIRONMENT
apply the given config environment
-f start a fastcgi server instead of the default HTTP
server
-s start a scgi server instead of the default HTTP server
-i IMPORTS, --import=IMPORTS
specify modules to import
-p PIDFILE, --pidfile=PIDFILE
store the process id in the given file
As far as an init.d script goes I think there are examples that can be Googled.
And the cherryd is found in your:
virtualenv/lib/python2.7/site-packages/cherrypy/cherryd
or in: https://bitbucket.org/cherrypy/cherrypy/src/default/cherrypy/cherryd
I wrote a tutorial/project skeleton, cherrypy-webapp-skeleton, which goal was to fill the gaps for deploying a real-world CherryPy application on Debian* for a web-developer. It features extended cherryd for daemon privilege drop. There's also a number of important script and config files for init.d, nginx, monit, logrotate. The tutorial part describes how to put things together and eventually forget about it. The skeleton part proposes a way of possible arrangement of CherryPy webapp project assets.
* It was written for Squeeze but practically it should be same for Wheezy.
Info on Daemonizer options
When using Daemonizer, the docs don't state the options, e.g. how to redirect stdout or stderr. From the source of the Daemonizer class you can find the options. As a reference take this example from my project:
# run server as a daemon
d = Daemonizer(cherrypy.engine,
stdout='/home/pi/Gate/log/gate_access.log',
stderr='/home/pi/Gate/log/gate_error.log')
d.subscribe()