We Keep Coding

How to connect to Azure Devops git repo through databricks?

I want to create a python notebook in databricks that will do the following -
Connect to Azure Devops Git repo
Make couple of changes in a yaml file
Commit the changes in the master branch
Push the changes back to repo
I tried the below code to achieve step 1:
import git
repo_url = <repo url>
pat_token = <token>
dbfs_path ='<dbfs_path>'
repo = git.Repo.clone_from(repo_url, dbfs_path, branch='master')
But I got the below error:
GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git clone --branch=master -v <repo_url> <dbfs_path>
stderr: 'Cloning into <dbfs_path>...
fatal: could not read Username for <repo_url>: No such device or address
I am not sure where to provide the username.

Python ldap3 connection fails when django app is deployed on ecs, but not on local

I have a django api that streams data from an Active directory source and processes it. My connection looks something like this
from ldap3 import Server, Connection
server = Server(url, get_info=ALL)
conn = Connection(server, username, password, auto_bind=True)
I put this app on a container, the dockerfile is simple and looks like this
FROM python:3.9
EXPOSE 8002
# Install Dependencies
ADD requirements.txt .
RUN pip install -r requirements.txt
ADD . .
CMD ./server.sh
Server.sh is also fairly simple:
#!/usr/bin/env bash
aws s3 cp s3://some_creds .
python manage.py makemigrations && python manage.py migrate && python manage.py runserver 0.0.0.0:80
Now, on local, and on gitpod this connection has no issues. I go on to do searches on the conn without problems. However, when I deploy the same container on ecs via ecr I was running on local, I get this error:
<class 'ldap3.core.exceptions.LDAPSocketOpenError'>, LDAPSocketOpenError('socket connection error while opening: [Errno 110] Connection timed out'), ('xxx.xxx.xxx.xxx', xxx))])
This may be a side effect of accessing the api from ssl, but if that is the case, I simply cannot replicate it on locale.
This error occurs only on POST requests; any other request goes through as expected.

The problem was caused by the Active Directory being only accessible via vpn/proxy. The gitpod instance I was using was company provided so it also had network access to the AD.
This was solved by using a VPC that had default access to this network.

Connection refused when tried to clone from github Enterprise

An error I met when I use git API
from git import Repo
from github import Github
Connecting to git.xxxxx.com (port 9418) ... fatal: unable to connect to git.xxxxx.com:
git.xxxxx.com[0: 141.113.0.105]: errno=Connection refused
I can clone code from Enterprise github by command
git clone https://[name]:[token]#git.git.xxxx.com/xxx.git
But I'm confusing the way of clone in git API
def walk_githubprojects(token, organization):
client = Github(base_url='https://git.xxx.com/api/v3', login_or_token=token)
user=client.get_user().get_repos()
for repo in client.get_organization(organization).get_repos():
print(repo.name)
print(repo.git_url)
Repo.clone_from(repo.git_url, 'my_path')
I can get repo names under organization, the script should clone repos to my local dir

The solution is：
from git import Repo
from github import Github
def walk_githubprojects(token, organization):
client = Github(base_url='https://git.xxxxx.com/api/v3', login_or_token=token)
user=client.get_user().get_repos()
for repo in client.get_organization(organization).get_repos():
HTTPS_REMOTE_URL = f'https://[name]:[token]#git.xxxxx.com/{repo.full_name}'
Repo.clone_from(HTTPS_REMOTE_URL, f'/xxxx/{repo.name}')

Firebase-admin:- Deployment Failed: ERROR: Your requirements.txt is invalid. Snapshot your logs for details.

I'm using python-flask and firebase-admin (for authentication) in my mobile app backend. I'm deploying my code to AWS Elastic Beanstalk. Everything is fine until I install the firebase-admin through "pip install firebase-admin".
I've committed changed to git.
Now, the deployment fails and displays the following message.
*MacBook-Pro:pets-friend-api santosh.guruju$ eb deploy
WARNING: Git is in a detached head state. Using branch "default".
WARNING: Git is in a detached head state. Using branch "default".
WARNING: Git is in a detached head state. Using branch "default".
WARNING: Git is in a detached head state. Using branch "default".
WARNING: Git is in a detached head state. Using branch "default".
Creating application version archive "app-d517-170725_142037".
Uploading PetsFrenzAPI/app-d517-170725_142037.zip to S3. This may take a while.
Upload Complete.
INFO: Environment update is starting.
INFO: Deploying new version to instance(s).
ERROR: Your requirements.txt is invalid. Snapshot your logs for details.
ERROR: [Instance: i-054100c8ffb51643c] Command failed on instance. Return code: 1 Output: (TRUNCATED)...)
File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1.
Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
ERROR: Unsuccessful command execution on instance id(s) 'i-054100c8ffb51643c'. Aborting the operation.
ERROR: Failed to deploy application.*

I've Fixed it!
As it states the problem is in Requirements.txt.
Instead of>> pip freeze > requirement.txt
Just added limited resources in the requirements file without version.
fix:requirement.txt contains
flask
flask_sqlalchemy
firebase_admin
pymysql

Fabric git pushing to wrong ip address

I'm working on win7 and trying to use fabric to push changes to an ubuntu 16.04 VPS. So far I have:
env.roledefs = {
'test': ['localhost'],
'dev': ['user#dev.example.com'],
'production': ['deploy#xxx.xx.xx.xx']
}
#roles('production')
def dir():
env.key_filename = '~/.ssh/id_rsa'
local("pip freeze > requirements.txt")
local("git add . --all && git commit -m 'fab'")
local("git push myproject master")
run('pwd')
...
When I run this the output is:
$ fab dir
[deploy#xx.xx.xx.xx] Executing task 'dir'
[localhost] local: pip freeze > requirements.txt
[localhost] local: git add . --all && git commit -m 'fab'
warning: LF will be replaced by CRLF in .idea/workspace.xml.
The file will have its original line endings in your working directory.
[master warning: LF will be replaced by CRLF in .idea/workspace.xml.
The file will have its original line endings in your working directory.
256de92] 'fab'
warning: LF will be replaced by CRLF in .idea/workspace.xml.
The file will have its original line endings in your working directory.
3 files changed, 10 insertions(+), 9 deletions(-)
[localhost] local: git push example master
debug1: Connecting to 198.91.88.101 [198.91.88.101] port 22.
debug1: connect to address 198.91.88.101 port 22: Connection refused
ssh: connect to host 198.91.88.101 port 22: Bad file number
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Fatal error: local() encountered an error (return code 128) while executing 'git push example master'
So fabric is trying to push to the wrong target ip address (this was an old vps address . I no longer have it.) I got rid of the VPS but saved the public and private key and uploaded the pub key to my new vps at a new ip address
The problem is I'm not sure where the old target address is being set. Is this a git issue. How do I redirect fabric to push to #roles('production')
When I look in my .ssh/known_hosts I see 198.91.88.101. So I'm wondering if that is involved in some way.

It is in the git remote configuration. Verify with git remote --verbose.