I was building a documentation site for my python project using mkdocstrings.
For generating the code referece files I followed this instructions https://mkdocstrings.github.io/recipes/
I get these errors:
INFO
- Building documentation... INFO
- Cleaning site directory INFO
- The following pages exist in the docs directory, but are not included in the "nav" configuration: - reference\SUMMARY.md
- reference_init_.md
... ...
- reference\tests\manual_tests.md ERROR
- mkdocstrings: No module named ' ' ERROR
- Error reading page 'reference/init.md': ERROR
- Could not collect ' '
This is my file structure:
This is my docs folder:
I have the same gen_ref_pages.py file shown in the page:
from pathlib import Path
import mkdocs_gen_files
nav = mkdocs_gen_files.Nav()
for path in sorted(Path("src").rglob("*.py")):
module_path = path.relative_to("src").with_suffix("")
doc_path = path.relative_to("src").with_suffix(".md")
full_doc_path = Path("reference", doc_path)
parts = tuple(module_path.parts)
if parts[-1] == "__init__":
parts = parts[:-1]
elif parts[-1] == "__main__":
continue
nav[parts] = doc_path.as_posix() #
with mkdocs_gen_files.open(full_doc_path, "w") as fd:
ident = ".".join(parts)
fd.write(f"::: {ident}")
mkdocs_gen_files.set_edit_path(full_doc_path, path)
with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: #
nav_file.writelines(nav.build_literate_nav()) # ```
This is my mkdocs.yml:
``` site_name: CA Prediction Docs
theme:
name: "material"
palette:
primary: deep purple
logo: assets/logo.png
favicon: assets/favicon.png
features:
- navigation.instant
- navigation.tabs
- navigation.expand
- navigation.top
# - navigation.sections
- search.highlight
- navigation.footer
icon:
repo: fontawesome/brands/git-alt
copyright: Copyright © 2022 - 2023 Ezequiel González
extra:
social:
- icon: fontawesome/brands/github
link: https://github.com/ezegonmac
- icon: fontawesome/brands/linkedin
link: https://www.linkedin.com/in/ezequiel-gonzalez-macho-329583223/
repo_url: https://github.com/ezegonmac/TFG-CellularAutomata
repo_name: ezegonmac/TFG-CellularAutomata
plugins:
- search
- gen-files:
scripts:
- docs/gen_ref_pages.py
- mkdocstrings
nav:
- Introduction: index.md
- Getting Started: getting-started.md
- API Reference: reference.md
# - Reference: reference/
- Explanation: explanation.md
i have the following yaml file:
rule_groups:
- name: "APPSTREAM"
allowed-domains:
- ".github.com"
- ".google.com"
source: "10.143.80.0/24"
- name: "TEST"
allowed-domains:
- ".microsoft.com"
- ".amazonaws.com"
source: "10.143.70.0/24"
i am calling inside a python script :
#!/usr/bin/env python3
import yaml
with open('settings.yaml', 'rb') as f:
config = yaml.safe_load(f)
core_config = config['rule_groups']
for workload in core_config:
print(workload)
{'name': 'APPSTREAM', 'allowed-domains': ['.github.com', '.google.com'], 'source': '10.143.80.0/24'}
{'name': 'TEST', 'allowed-domains': ['.microsoft.com', '.amazonaws.com'], 'source': '10.143.70.0/24'}
i am trying to create a dynamic string for every name , allowed-domains and sid and write the outpot into a file , as following:
$APPSTREAM ,.google.com, sid:1
$APPSTREAM ,.github.com, sid:2
$TEST, .microsoft.com, sid:3
$TEST, .amazonaws.com,sid:4
any help will be appreciated
solution
#!/usr/bin/env python3
import yaml
with open('settings.yaml', 'rb') as f:
config = yaml.safe_load(f)
core_config = config['rule_groups']
i = 0
for workload in core_config:
for domain in (workload['allowed-domains']):
i = i+1
print(f"${workload['name']},'{domain}',sid:{i}")
it seems to be a problem on where python is searching for de library and not finding it.
still, im very new in this so may be its another thing.
this is the error (i separated the middle part where i think shows the problem):
ftuser#a5a1d3ed08d3:/freqtrade$ freqtrade backtesting --strategy canal
2022-08-26 03:51:37,394 - freqtrade.configuration.load_config - INFO - Using config: user_data/config.json ...
2022-08-26 03:51:37,483 - freqtrade.loggers - INFO - Verbosity set to 0
2022-08-26 03:51:37,484 - freqtrade.configuration.configuration - INFO - Using max_open_trades: 1 ...
2022-08-26 03:51:37,716 - freqtrade.configuration.configuration - INFO - Using user-data directory: /freqtrade/user_data ...
2022-08-26 03:51:37,718 - freqtrade.configuration.configuration - INFO - Using data directory: /freqtrade/user_data/data/binance ...
2022-08-26 03:51:37,719 - freqtrade.configuration.configuration - INFO - Parameter --cache=day detected ...
2022-08-26 03:51:37,719 - freqtrade.configuration.check_exchange - INFO - Checking exchange...
2022-08-26 03:51:37,741 - freqtrade.configuration.check_exchange - INFO - Exchange "binance" is officially supported by the Freqtrade development team.
2022-08-26 03:51:37,741 - freqtrade.configuration.configuration - INFO - Using pairlist from configuration.
2022-08-26 03:51:37,741 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2022-08-26 03:51:37,746 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Backtesting mode
2022-08-26 03:51:37,746 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2022-08-26 03:51:37,746 - freqtrade.exchange.exchange - INFO - Using CCXT 1.92.20
2022-08-26 03:51:37,746 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'future'}}
2022-08-26 03:51:37,766 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'future'}}
2022-08-26 03:51:37,782 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance"
2022-08-26 03:51:39,052 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...
2022-08-26 03:51:39,097 - freqtrade.resolvers.iresolver - WARNING - Could not import /freqtrade/user_data/strategies/canal.py due to 'cannot import name 'SSLchannels' from 'technical.indicators' (/home/ftuser/.local/lib/python3.10/site-packages/technical/indicators/init.py)'
2022-08-26 03:51:39,182 - freqtrade - ERROR - Impossible to load Strategy 'canal'. This class does not exist or contains Python code errors.
2022-08-26 03:51:39,182 - freqtrade.exchange.exchange - INFO - Closing async ccxt session.
this is the code in VS Code:
import numpy as np # noqa
import pandas as pd # noqa
from pandas import DataFrame
from freqtrade.strategy import (BooleanParameter, CategoricalParameter, DecimalParameter,
IStrategy, IntParameter)
# --------------------------------
# Add your lib to import here
import talib.abstract as ta
import freqtrade.vendor.qtpylib.indicators as qtpylib
from technical.indicators import SSLchannels
# This class is a sample. Feel free to customize it.
class canal(IStrategy):
INTERFACE_VERSION = 3
# Can this strategy go short?
can_short: bool = False
# Minimal ROI designed for the strategy.
# This attribute will be overridden if the config file contains "minimal_roi".
minimal_roi = {
"60": 0.01,
"30": 0.02,
"0": 0.04
}
# Optimal stoploss designed for the strategy.
# This attribute will be overridden if the config file contains "stoploss".
stoploss = -0.10
# Trailing stoploss
trailing_stop = False
# trailing_only_offset_is_reached = False
# trailing_stop_positive = 0.01
# trailing_stop_positive_offset = 0.0 # Disabled / not configured
# Optimal timeframe for the strategy.
timeframe = '5m'
# Run "populate_indicators()" only for new candle.
process_only_new_candles = True
# These values can be overridden in the config.
use_exit_signal = True
exit_profit_only = False
ignore_roi_if_entry_signal = False
buy_rsi = IntParameter(low=1, high=50, default=30, space='buy', optimize=True, load=True)
sell_rsi = IntParameter(low=50, high=100, default=70, space='sell', optimize=True, load=True)
short_rsi = IntParameter(low=51, high=100, default=70, space='sell', optimize=True, load=True)
exit_short_rsi = IntParameter(low=1, high=50, default=30, space='buy', optimize=True, load=True)
# Number of candles the strategy requires before producing valid signals
startup_candle_count: int = 30
# Optional order type mapping.
order_types = {
'entry': 'limit',
'exit': 'limit',
'stoploss': 'market',
'stoploss_on_exchange': False
}
# Optional order time in force.
order_time_in_force = {
'entry': 'gtc',
'exit': 'gtc'
}
plot_config = {
'main_plot': {
'tema': {},
'sar': {'color': 'white'},
},
'subplots': {
"MACD": {
'macd': {'color': 'blue'},
'macdsignal': {'color': 'orange'},
},
"RSI": {
'rsi': {'color': 'red'},
}
}
}
def informative_pairs(self):
"""
Define additional, informative pair/interval combinations to be cached from the exchange.
These pair/interval combinations are non-tradeable, unless they are part
of the whitelist as well.
For more information, please consult the documentation
:return: List of tuples in the format (pair, interval)
Sample: return [("ETH/USDT", "5m"),
("BTC/USDT", "15m"),
]
"""
return []
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
# RSI
dataframe['rsi'] = ta.RSI(dataframe)
return dataframe
def populate_entry_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[
(
# Signal: RSI crosses above 30
(qtpylib.crossed_above(dataframe['rsi'], self.buy_rsi.value)) &
(dataframe['volume'] > 0) # Make sure Volume is not 0
),
'enter_long'] = 1
dataframe.loc[
(
# Signal: RSI crosses above 70
(qtpylib.crossed_above(dataframe['rsi'], self.short_rsi.value)) &
(dataframe['volume'] > 0) # Make sure Volume is not 0
),
'enter_short'] = 1
return dataframe
def populate_exit_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[
(
# Signal: RSI crosses above 70
(qtpylib.crossed_above(dataframe['rsi'], self.sell_rsi.value)) &
(dataframe['volume'] > 0) # Make sure Volume is not 0
),
'exit_long'] = 1
dataframe.loc[
(
# Signal: RSI crosses above 30
(qtpylib.crossed_above(dataframe['rsi'], self.exit_short_rsi.value)) &
# Guard: tema below BB middle
(dataframe['volume'] > 0) # Make sure Volume is not 0
),
'exit_short'] = 1
return dataframe
I left RSI indicator so that i could coment:
#from technical.indicators import SSLchannels
and test the code is ok and it works. It runs the backtest ok.
Heres how i have the folders in my PC
I also tryed choosing python 3.8 and 3.10 in VS Code just ro try and both work well if i take out technical library and shows error if i put it.
any help would be apreciated.
Thanks!
I would think that either you need to pip install the technical.indicators or use docker. I prefer using docker to execute freqtrade commands as they already have all the dependencies installed.
I have a list of yaml files which I would like to edit in the same way. Put the following block under spec.template.imagePullSecret and spec.template.container.imagePullPolicy in each file:
imagePullSecrets:
- dockerpullsecret
container:
imagePullPolicy: IfNotPresent
This is an example of one file:
apiVersion: argoproj.io/v1alpha1
kind: Sensor
meta data:
name: sensor-finanda-ci
namespace: argo-events
spec:
template:
serviceAccountName: argo-events-sa
dependencies:
- name: eventsource-iv
eventSourceName: eventsource-iv
eventName: iv
This is my desired output:
apiVersion: argoproj.io/v1alpha1
kind: Sensor
meta data:
name: sensor-finanda-ci
namespace: argo-events
spec:
template:
serviceAccountName: argo-events-sa
imagePullSecrets:
- dockerpullsecret
container:
imagePullPolicy: IfNotPresent
dependencies:
- name: eventsource-iv
eventSourceName: eventsource-iv
eventName: iv
If your heart is set on a Python solution, something like this might do:
import os
import yaml
for yaml_filename in [filename for filename in os.listdir('.') if filename.endswith('.yaml')]:
with open(yaml_filename, 'r') as yaml_file:
yaml_obj = yaml.safe_load(yaml_file)
if 'spec' not in yaml_obj:
yaml_obj['spec'] = {"template": {}}
if 'template' not in yaml_obj['spec']:
yaml_obj['spec']['template'] = {"imagePullSecrets": [], "container": {}}
if 'container' not in yaml_obj['spec']['template']:
yaml_obj['spec']['template']['container'] = {}
yaml_obj['spec']['template']['imagePullSecrets'] = ["dockerpullsecret"]
yaml_obj['spec']['template']['container']['imagePullPolicy'] = 'IfNotPresent'
with open(yaml_filename, 'w') as yaml_file:
yaml.dump(yaml_obj, yaml_file)
If you're open to command-line utils, this is more concise:
Loop over the files and use yq to edit them in-place.
for file in *.yaml; do
yq -i e '.spec.template.imagePullSecrets = ["dockerpullsecret"] | .spec.template.container.imagePullPolicy = "IfNotPresent"' "$file"
done
we are getting output in this format
2020-11-19 12:00:01,414 - INFO - clusterDC -Backup started
2020-11-19 12:00:01,415 - Debug - executing command: /opt/couchbase/bin/cbbackupmgr backup --archive /backup/clusterDC --repo clusterDC_date --cluster nodedc --username user --threads 16
2020-11-19 12:00:01,414 - INFO - clusterDC - Backup Succeeded. Backup successfully completed.
But now we want them in below JSON format.
"backup":[
{
"server_name":"nodedc",
"status":"Success/Failed",
"backup_start_time":"yyyy-mm-dd hh:mm:ss.mmmuu",
"cluster_name":"clusterDC",
"location":"/backup/clusterDc/clusterDC_date",
"backup_end_time":"yyyy-mm-dd hh:mm:ss:mmmuu"
}
There are multiple ways you could do this. One would be to parse the your output into a a list then use numpy to convert that to json. For parsing, just use string manipulation (maybe regex if needed) to populate your list.
If the output follows the same format each time:
#output = raw output
output = [line.split() for line in output.splitlines()]
backup = {}
backup["server_name"] = text[1][14]
backup["status"] = text[2][8]
backup["backup_start_time"] = f"{text[0][0]} {text[0][1].replace(',', ':')}"
backup["cluster_name"] = text[0][5]
backup["location"] = f"{text[1][10]}/{text[1][12]}"
backup["backup_end_time"] = f"{text[2][0]} {text[2][1].replace(',', ':')}"
backup = {'backup': [backup]}
print(backup)
Output:
{'backup': [{'server_name': 'nodedc', 'status': 'Succeeded.', 'backup_start_time': '2020-11-19 12:00:01:414', 'cluster_name': 'clusterDC', 'location': '/backup/clusterDC/clusterDC_date', 'backup_end_time': '2020-11-19 12:00:01:414'}]}