We Keep Coding

SQL Server stored procedure Python passed file path

I am trying to pass a file directory to a Python script in order to process the contents. Here is the EXEC statement and below my code is the error. It is supposed open the file and do some processing.
EXEC ScrapeData 'F:\FinancialResearch\SEC\myEdgar\sec-edgar-filings\WIRE\10-Q\0001753926-20-000110\full-submission.txt'
DROP PROCEDURE IF EXISTS ScrapeData;
GO
CREATE PROCEDURE [dbo].[ScrapeData]
(#filepath varchar(300))
AS
BEGIN
EXEC sp_execute_external_script
#language = N'Python',
#script = N'
import html2text
from bs4 import BeautifulSoup
import pyodbc
import re
import requests
import pandas as pd
with open(#filepath) as f:
contents = f.read()
soup = BeautifulSoup(contents, ''lxml'')
.....
Error:
Msg 39004, Level 16, State 20, Line 0
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 0
An external script error occurred:
Error in execution. Check the output for more information.
Traceback (most recent call last):
File "", line 3, in
File "E:\ProgramData\REAPER\Temp-PY\Appcontainer1\DE944528-CBDA-4AFD-B50F-32AD2CFD35B7\sqlindb_0.py", line 45
with open(#filepath) as f:
^
SyntaxError: invalid syntax
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
SqlSatelliteCall function failed. Please see the console output for more information.

...for a file c:\temp\test.txt..
declare #myfile varchar(100) = 'c:\temp\test.txt';
EXEC sp_execute_external_script #language = N'Python', #script = N'
print(filename)
print(xyz)
print(paramZ)
num_lines = sum(1 for line in open(filename))
print("------")
print(num_lines)
',
#params= N'#filename varchar(100), #paramZ int, #xyz varchar(10)', #filename = #myfile, #xyz='abcd', #paramZ = 100;

ZeroMQ threw ZMQError on socket bind to an ipc:// protocol address (python)

I'm trying to use the IPC protocol with ZeroMQ in Python.
import sys
import time
from random import randint
import zmq
def main(url=None):
ctx = zmq.Context.instance()
publisher = ctx.socket(zmq.PUB)
if url:
publisher.bind(url)
else:
publisher.bind('ipc://var/run/fast-service')
# Ensure subscriber connection has time to complete
time.sleep(1)
# Send out all 1,000 topic messages
for topic_nbr in range(1000):
publisher.send_multipart([
b"%03d" % topic_nbr,
b"Save Roger",
])
if __name__ == '__main__':
main(sys.argv[1] if len(sys.argv) > 1 else None)
It gives the following error:
Traceback (most recent call last):
File "pathopub.py", line 43, in <module>
main(sys.argv[1] if len(sys.argv) > 1 else None)
File "pathopub.py", line 19, in main
publisher.bind("ipc://var/run/fast-service")
File "zmq/backend/cython/socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
zmq.error.ZMQError: No such file or directory for ipc path "var/run/fast-service".
I don't understand why would this happen with the socket.bind() function, since in the documentation it says:
When binding a socket to a local address using zmq_bind() with the ipc transport, the endpoint shall be interpreted as an arbitrary string identifying the pathname to create.
which means that it is not necessary to supply an already created directory.

The URL scheme is ipc://. You need to add an absolute path /var/run/fast-service. So,
publisher.bind('ipc:///var/run/fast-service')
More generally the URL is ipc://<host>/<path>. You want local host, so that part is empty. A file system URL is similar, file:///home/foo/bar.txt references /home/foo/bar.txt on the local host.

Script working on Windows, but not on Linux?

I am learning to program in Python and am still at the very beginning.
I wrote a 2 scripts to cut out IP-addresses from a nmap-output.
The first script cuts out the IP-addresses:
import re
file = open("test.txt", "r")
ips = open("ips.txt", "w+")
for line in file:
ip = re.findall(r'[0-9]+(?:\.[0-9]+){3}', line)
if "filtered" in line:
ips.write(str(ip) + "\n")
This code works fine on Windows and Linux, but (I hope I'm right) the for-loop gets every line as a list. That means, my IP-addresses have the format ['x.x.x.x'].
I wrote a second script to delete all the unnecessary characters ([ ' and ]):
ip = open("ips.txt", "r")
ip_filtered = open("ip_filtered.txt", "w")
for line in ip:
s = line
neu = s.translate(({ord(i): None for i in '[\']'}))
ip_filtered.write(str(neu))
This script works well on Windows (I got a new file, just with IP-addresses), but on Linux I get the following error:
Traceback (most recent call last):
File "zeichen_loeschen.py", line 6, in <module>
neu = s.translate(({ord(i): None for i in '[\']'}))
TypeError: expected a string or other character buffer object
What's the reason for this error?
Thanks in advance :)

I get the same message when running with the python command on linux (which uses python 2.7). Try running it with the python3 command and your code should work.

Python 3 permission error when playing a sound file (mp3, playsound module)

The program was working fine a few days ago, and it just stopped today. Not a single letter has been changed. One of my troubleshooting steps was to remove the file 'output1.mp3' and check if it will work that way, but it didn't. Another thing is that when it wasn't printing out the error, it would continue to play just this one sound file, whether or not it said the correct thing. Here's the latest error I got:
Traceback (most recent call last):
File "main3.py", line 123, in <module>
start()
File "main3.py", line 117, in start
tts(say)
File "main3.py", line 24, in tts
play('output1.mp3')
File "C:\Program Files (x86)\Python36-32\lib\site-packages\playsound.py", line 35, in _playsoundWin
winCommand('open "' + sound + '" alias', alias)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\playsound.py", line 31, in winCommand
raise PlaysoundException(exceptionMessage)
playsound.PlaysoundException:
Error 275 for command:
open "output1.mp3" alias playsound_0.8842337577803419
Cannot find the specified file. Make sure the path and filename are correct.
Here's the code that I use:
import boto3 # used to 'pythonize' Amazon Polly TTS
import speech # speech recognition
from playsound import playsound as play # for playing sound files
import sys # basically only for exiting
# import locator # for determining the location of the user based on IP address
import locator2 # for confirming auto-detected location
# import locator3 # for definitely confirming auto-detection location
import question # module for answering any question
from datetime import datetime # for displaying the date and time
# from time import sleep # sleep (wai()t for) function
from os import popen as read # for reading command outputs "read('COMMAND').read()"
def tts(text):
polly = boto3.client("polly")
spoken_text = polly.synthesize_speech(Text=str(text),
OutputFormat='mp3',
VoiceId='Brian')
with open('output11.mp3', 'wb') as f:
f.write(spoken_text['AudioStream'].read())
f.close()
play('output11.mp3')
def ask(query):
question.question(query)
response = question.answer
print(response)
tts(response)
ai()
def time():
now = datetime.now()
print("Current date and time: ")
print(now.strftime("%H:%M") + "\n")
tts("It is " + now.strftime("%H:%M"))
ai()
def weather():
response = "Based on your IP address, I've detected that you're located in %s. Is that correct?" % locator2.city
print(response)
tts(response)
speech.speech()
if 'yes' in speech.x:
z = read('weather "%s"' % locator2.city).read()
print(z)
tts(z)
ai()
else:
response = 'Please say the name of the city you would like the weather information for. \n'
print(response)
tts(response)
speech.speech()
city = speech.x
wdr = read('weather "%s"' % city).read()
print(wdr)
tts(wdr)
ai()
def thank():
response = "You're very welcome! \n"
print(response)
tts(response)
ai()
def ext():
response = "Goodbye!"
print(response)
tts(response)
sys.exit()
def error():
response = "Invalid request detected, please try again...\n"
print(response)
tts(response)
ai()
def ai():
print('Started listening - Speak!')
speech.speech()
spoken = speech.x
# TODO new commands should be written above this, and their trigger words below :)
question_words = ['?', 'what', 'where', 'when', 'who', 'how', 'why']
if 'time' in spoken:
time()
elif 'weather' in spoken:
weather()
elif any(word in spoken for word in question_words):
ask(spoken)
elif 'thank' in spoken:
thank()
elif 'exit' or 'quit' or 'deactivate' in spoken:
ext()
else:
error()
def start():
say = "Hello! My name is Dave, and I'm your personal assistant. How may I help you today? \n"
print(say)
tts(say)
ai()
if __name__ == '__main__':
try:
start()
except KeyboardInterrupt:
ext()
The speech synthesizer is Amazon Polly. By the way, I was using PyCharm as an IDE and working on Windows 10. When I switch to my Linux machine the speech recognition part breaks.
UPDATE: I was tweaking the code a bit and managed to fix the pyaudio error, but I got another one in the process, this time it was about permissions. Here's the error log:
Traceback (most recent call last):
File "C:/Users/Despot/Desktop/DAv3/main3.py", line 123, in <module>
start()
File "C:/Users/Despot/Desktop/DAv3/main3.py", line 118, in start
ai()
File "C:/Users/Despot/Desktop/DAv3/main3.py", line 96, in ai
time()
File "C:/Users/Despot/Desktop/DAv3/main3.py", line 39, in time
tts("It is " + now.strftime("%H:%M"))
File "C:/Users/Despot/Desktop/DAv3/main3.py", line 21, in tts
with open('output11.mp3', 'wb') as f:
PermissionError: [Errno 13] Permission denied: 'output11.mp3'
UPDATE 2: I have been tikering about and I've found that the issue is only present on my Windows 10 machine, the program works fine on Linux.

Try using the absolute path (complete path) of the audio file instead of the relative path.
For example: "C:/Users/Adam/Desktop/dolphin.wav" instead of just "dolphin.wav"
This worked for me.

playsound libs has a windows directories in them.
If this fail only on Linux you should install playsound lib on the linux machine and then copy only the main3.py to it.

Answer for UPDATE2:
Copy output11.mp3 to another location and change the path to the new location:
with open('CHANGE-THIS-PATH', 'wb') as f:
Also make sure python run as administrator.

I solved this problem by moving the .py and .wav files to a folder less deep inside the file system.

I just renamed the audio file from sample.wav to sample
and then run it as playsound('sample.wav')
Luckily It ran.
Then I came to know that previously it was stored as sample.wav.wav.
So let's keep your audio file name as output.wav and try running your audio file as:
playsound('output.wav.wav') # or else rename it to output and run as
playsound('output.wav') # likewise for other audio formats too

Python import error

I am trying to run a python file from the command line with a single parameter in Ubuntu 12.04. The program works if I simply run it from the IDE and pass the parameter in the code. However, if I call 'python readFromSerial1.py 3' in the command prompt, I get:
Traceback (most recent call last):
File "readFromSerial1.py", line 62, in <module>
main()
File "readFromSerial1.py", line 6, in main
readDataFromUSB(time)
File "readFromSerial1.py", line 9, in readDataFromUSB
import usb.core
ImportError: No module named usb.core
I'm a little confused as the module imports correctly if I run from the IDE. I download the pyUSB module and extracted it (its filename is pyusb-1.0.0a3). I then copied this file into
/usr/local/lib/python2.7/site-packages/. Is that the correct procedure? I have a feeling the issue is due to python simply not being able to find the usb module and I need to put it in the correct location. My code is below, and any help would be greatly appreciated:
readFromSerial1.py
import sys
def main():
time = sys.argv[1]
#time = 1
readDataFromUSB(time)
def readDataFromUSB(time):
import usb.core
#find device
dev = usb.core.find(idVendor=0x099e, idProduct=0x0001) #GPS info
#Was the device found?
if dev is None:
raise ValueError('Device not found')
else:
print "Device found!"
#Do this to avoid 'Errno16: Resource is busy'
if dev.is_kernel_driver_active(0):
try:
dev.detach_kernel_driver(0)
except usb.core.USBError as e:
sys.exit("Could not detach kernel driver: %s" % str(e))
#Sets default config
dev.set_configuration()
#Gets default endpoint
endpoint = dev[0][(0,0)][0]
writeObject = open("InputData.txt", "w")
#iterate for time purposes
for i in range(0, (time*6)): #sys.argv is command line variable for time input
data = dev.read(endpoint.bEndpointAddress, endpoint.wMaxPacketSize, 0, 100000)
sret = ''.join([chr(x) for x in data])
writeObject.write(sret);
print sret
'''
newStr = ''.join(sret[7:14])
compareStr = ",*4F"
if (newStr == compareStr):
print "The GPS is not reading in any values right now. Try going somewhere else with better reception."
else:
print sret[7:14]
'''
writeObject.close()
main()