We Keep Coding

Extract zip file inline in Oracle OCI - Object Storage without downloading to save time

Is it possible to extract a zip file 'inline' which is in cloud say Oracle cloud, Object storage. Meaning, without downloading it and extracting it in the o/s and re-uploading it to object storage, because the file size is big and we need to save time on upload/download.. Any sample code, with Oracle Functions, or python, java etc. ? Is it possible ? I tried with S3 browser/explorer or other similar tools, but that basically at the background, downloads and extract on local computer.

If I understand the question correctly, your use case would be that you have a compressed value on the server and want to extract it on the server and keep it there.
This is possible and mostly depends on how the values has been compressed.
If you use the Lempel-Ziv-Welch algorithm used in the UTL_COMPRESS package, you can extract it directly in PL/SQL.
For other formats like zip, you will need to use some custom Java code like the following example:
CREATE OR REPLACE
JAVA SOURCE NAMED ZIP_Java
AS
import java.io.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import java.util.zip.ZipInputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.BufferedOutputStream;
import java.sql.Blob;
public class Java_Class {
public static int ZipBlob(Blob inLob, Blob[] outLob, String filename) {
try {
// create the zipoutputstream from the end of the outLob
Blob zipLob = outLob[0];
OutputStream os = zipLob.setBinaryStream(1);
ZipOutputStream zos = new ZipOutputStream(os);
// add one zip entry
ZipEntry entry = new ZipEntry(filename);
zos.putNextEntry(entry);
// write data to the zip lob
long len = inLob.length();
long offset = 0;
byte[] buffer;
int chunksize = 32768;
while (offset < len) {
buffer = inLob.getBytes(offset + 1, chunksize);
if (buffer == null)
break;
zos.write(buffer, 0, buffer.length);
offset += buffer.length;
}
zos.closeEntry();
zos.close();
outLob[0] = zipLob;
} catch (Exception e) {
System.out.println("Exception: " + e.toString());
e.printStackTrace(System.out);
return 0;
}
return 1;
}
public static int UnzipBlob(Blob inLob, Blob[] outLob, String filename) {
try {
final int kBUFFER = 2048;
InputStream inps = inLob.getBinaryStream();
ZipInputStream zis = new ZipInputStream(inps);
ZipEntry entry;
Blob fileLob = outLob[0];
OutputStream os = fileLob.setBinaryStream(1);
while((entry = zis.getNextEntry()) != null) {
if (entry.getName().equalsIgnoreCase(filename)) {
byte data[] = new byte[kBUFFER];
BufferedOutputStream dest = new BufferedOutputStream(os, kBUFFER);
int count;
while ((count = zis.read(data, 0, kBUFFER)) != -1) {
dest.write(data, 0, count);
}
dest.flush();
dest.close();
}
}
zis.close();
return 1;
} catch (Exception e) {
System.out.println("Exception: " + e.toString());
e.printStackTrace();
return 0;
}
}
}
/
CREATE OR REPLACE
FUNCTION ZipBlobJava(theSource IN BLOB, theDestination IN OUT NOCOPY BLOB, theFilename IN VARCHAR2) RETURN NUMBER
AS LANGUAGE JAVA NAME 'Java_Class.ZipBlob(java.sql.Blob, java.sql.Blob[], java.lang.String) return int';
/
CREATE OR REPLACE
FUNCTION UnzipBlobJava(theSource IN BLOB, theDestination IN OUT NOCOPY BLOB, theFilename IN VARCHAR2) RETURN NUMBER
AS LANGUAGE JAVA NAME 'Java_Class.UnzipBlob(java.sql.Blob, java.sql.Blob[], java.lang.String) return int';
/

Issue packing / unpacking message for UART between ESP-01S and Raspberry Pi Pico

My end goal is to set up a Raspberry Pi Pico with an ESP-01S to enable wifi. The Pico will periodically check in with the server and put the ESP to sleep when not in use. Communication between the two is over UART. I want to support both a GET and POST HTTP request from the ESP.
This is the message structure I am using between the two devices.
| Always start with MSG | Request Type | Next message Size
| | Next message size | | | URL
V V V V V
|-----|-------------------|-----|-------------------|-----------|
[M|S|G|\x00\|x00\|x00\|x03|G|E|T|\x00|\x00|\x00|\x1f|h|t|t|p|...]
[M|S|G|\x00\|x00\|x00\|x04|P|O|S|T|URL SIZE...|URL...|\x00|\x00|\x006|POST Data...]
|-----|-------------------|-------|-----------|------|---------------|------------|
^ ^
| | Post Data
| Post data size
For testing purposes I am generating the strings in python, printing them and pasting them directly in the .cpp file I'm flashing onto the ESP.
Here is the snippet of code I'm using on my pc to generate the message.
import struct
import json
url = "http://192.168.X.X:8090/korok"
size_of_url = struct.pack('!I', len(url))
data = json.dumps({
"serial": "12345",
"sensor_data": {"0": 75, "1": 67}
})
size_of_data = struct.pack('!I', len(data))
print(f"{struct.pack('!I', len('GET'))}GET{size_of_url}{url}")
print(f"{struct.pack('!I', len('POST'))}POST{size_of_url}{url}{size_of_data}{data}")
>>>> ...
b'\x00\x00\x00\x03'GETb'\x00\x00\x00\x1f'http://192.168.X.X:8090/korok
b'\x00\x00\x00\x03'GETb'\x00\x00\x00\x1f'http://192.168.X.X:8090/korokb'\x00\x00\x006'{"serial": "12345", "sensor_data": {"0": 75, "1": 67}}
And here is the code running on the ESP. I left comments on some of the behaviors I'm experiencing.
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#define GPIO_STATUS 2
#define BUFFER_SIZE 256
#define DATA_SIZE 4
char buf[BUFFER_SIZE];
void setup() {...}
...
// Struct is used so I can also get the size to use as an offset
// and read out part of the message in the buffer.
struct Message
{
char *value;
unsigned int size;
unsigned int totalMessageSize;
};
Message readMessage(char *data)
{
struct Message message;
message.size = ntohl(*(unsigned int *)(data)); // Unable to read shorthand hex
Serial.println(message.size);
message.totalMessageSize = message.size + DATA_SIZE; // Shorthand is only 3 char
message.value = (char *)malloc(message.size);
Serial.println(message.size);
int idx = DATA_SIZE;
int jdx = 0;
while (idx < message.size + DATA_SIZE && idx < BUFFER_SIZE)
{
message.value[jdx++] = data[idx++];
}
return message;
}
void loop()
{
delay(3000);
char * msg = "\x00\x00\x00\x03GET\x00\x00\x00\x1fhttp://192.168.X.X:8090/korok";
// char *msg = read_message();
if (msg)
{
Serial.print("\n");
int bufferIdx = 0;
struct Message request = readMessage(msg);
bufferIdx = request.totalMessageSize;
// This is odd and doing it due to odd behavior with ntohl and a variable
// offset. See below.
memcpy(msg, msg + bufferIdx, BUFFER_SIZE - bufferIdx);
struct Message url = readMessage(msg);
struct Message data;
if (memcmp(request.value, "GET", 3) == 0)
{
}
else if (memcmp(request.value, "POST", 4) == 0)
{
bufferIdx = url.totalMessageSize;
memcpy(msg, msg + bufferIdx, BUFFER_SIZE - bufferIdx);
struct Message data = readMessage(msg);
Serial.println(data.value);
}
free(request.value);
free(url.value);
if (memcmp(request.value, "POST", 4) == 0)
{
free(data.value);
}
}
}
Here is one of the two issues, though I found a workaround by doing a memcpy of my original char* offsetting the beginning index. The first line below works but the second throws LoadStoreAlignmentCause exception.
Ideally I'd like to understand what's going on here and to get this working without the memcpy.
ntohl(*(unsigned int *)(msg + 7)); // Works
int offset = 7;
ntohl(*(unsigned int *)(msg + offset)); // Throws Exception (9) LoadStoreAlignmentCause
The main issue I'm experiencing is when I'm packing the size in python some hex values are being shorthanded. e.g struct.pack('!I', 54) == \x00\x00\x006
When this happens ntohl() seems to read an address it shouldn't and outputs 635. Also since I'm expecting four char the rest of the message is off by one index.
A few questions regarding this issue. What is the name of this shorthand hex syntax? Is there anyway to get python to not output this short hand? Or are there any suggestions on how to get this working on the ESP?

Change Rijndael decryption logic from C# to Python

I have some already encrypted data which needs to be decrypted using python. The decryption logic in C# looks as given below.
using System.Security.Cryptography;
private const string Url_ENCRYPTION_KEY = "abcd123456";
private readonly static byte[] URL_SALT = Encoding.ASCII.GetBytes(Url_ENCRYPTION_KEY.Length.ToString());
public static string Decrypt(string inputText) {
try {
if (!string.IsNullOrEmpty(inputText)) {
RijndaelManaged rijndaelCipher = new RijndaelManaged();
byte[] encryptedData = Convert.FromBase64String(inputText.Replace(" ", "+"));
PasswordDeriveBytes secretKey = new PasswordDeriveBytes(Url_ENCRYPTION_KEY, URL_SALT);
using (ICryptoTransform decryptor =
rijndaelCipher.CreateDecryptor(secretKey.GetBytes(32), secretKey.GetBytes(16))) {
using (MemoryStream memoryStream = new MemoryStream(encryptedData)) {
using (CryptoStream cryptoStream =
new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read)) {
byte[] plainText = new byte[encryptedData.Length];
int decryptedCount = cryptoStream.Read(plainText, 0, plainText.Length);
return Encoding.Unicode.GetString(plainText, 0, decryptedCount);
}
}
}
}
}
catch(Exception ex) {
//clsErrorHandler.LogError(ex);
}
return inputText;
}
I have tried libs like pprp and python's cryptography but the solutions out there use PBKDF2, while the C# code here supplies the decryptor bytes of key and salt as key and IV values.
From what I looked the PasswordDeriveBytes function basically work as a somewhat modified PBKDF1,
but all of the solutions I tried fail with somekind of esoteric this size doesn't match with that size errors.
Here is one implementation of PasswordDeriveBytes I found floating out there but I am at a loss on how to do something similar to secretKey.GetBytes(32) and creating a decryptor
import hashlib
from base64 import b64decode
def MS_PasswordDeriveBytes(pstring, salt, hashfunc, iterations, keylen):
if iterations > 0:
lasthash = hashlib.sha1(pstring+salt).digest()
iterations -= 1
else:
print("Iterations must be > 0")
#If iterations is 1 then basically the same thing happens as 2 based on my testing
#if iterations == 0 and keylen > len(lasthash):
#print("Dunno what happens here")
#return -1
for i in range(iterations-1):
lasthash = hashlib.sha1(lasthash)
bytes = hashlib.sha1(lasthash).digest()
ctrl = 1
while len(bytes) < keylen:
bytes += hashlib.sha1(str(ctrl)+lasthash).digest()
ctrl += 1
return(bytes[:keylen])
stpass = 'amp4Z0wpKzJ5Cg0GDT5sJD0sMw0IDAsaGQ1Afik6NwXr6rrSEQE='
slt = 'aGQ1Afik6NampDT5sJEQE4Z0wpsMw0IDAD06rrSswXrKzJ5Cg0G='
initv = '#1B2c3D4e5F6g7H8'
enc_str = b64decode('B5YDTLEDBjd+8zy5lzEfjw==')
derbytes = MS_PasswordDeriveBytes(stpass, slt, hashlib.sha1, iterations=2, keylen=32)

AES 128 how to brute force with partial key in python

I am beginner in python so I need help
I have encrypted file and partial key file ,the partial key file have 96 bit out of 128 so i need to guess
the rest of the key and decrypt the file ,the hint is the first word is "hello" in the encrypted
my questions:
how to guess the rest of key ?
how to check if the key is the right key and how much time did take?

Sorry that the source code in my answer is in Java but up to now I never wrote a single line in python. As you showed us nearly no self written code I assume you need a path to find a way to do it in python.
Some basics to start with - AES 128 encryption has a 128 bit or 16 bytes long key for encryption and later decryption. You told us that you already know 96 bits = 12 bytes so there are "just" 4 bytes left to guess. Each byte has a "capacity" of (integer) 256 so in maximum you would have to test 256 * 256 * 256 * 256 combinations = 4,294,967,296 combinations. Assuming that the key is of random you need half of this to try to find the real key so in real there are 2,147,483,648 combinations - a todays home computer does this within an hour or less.
Below you find my code and I setup an online compiler with the code - in this code I "restricted" the search for the last 2 bytes (so it assumes that 14 of the
16 bytes are known) because the online compiler would abort the program with a "running too long" runtime error.
The code is well commented and not optimized for speed but for readability. In the end you get an output like this one (as I'm using a random key and
initialization vector for AES in CBC mode your output will be different), the online compiler is available here: https://repl.it/#javacrypto/JavaAesCbc128BruteForce:
AES CBC String Encryption - brute force decryption
key length: 16 data: 0cb878f9da009be3ef487f54aa8c3230
keyGuess length: 16 data: 0cb878f9da009be3ef487f54aa8c0000
*** key found ***
key length: 16 data: 0cb878f9da009be3ef487f54aa8c3230
plaintext: hello this is just plaintext
code:
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
public class Main {
public static void main(String[] args) throws Exception {
System.out.println("AES CBC String Encryption - brute force decryption");
String plaintext = "hello this is just plaintext";
// this plaintext beginning is known
String plaintextKnown = "hello";
byte[] plaintextKnownByte = plaintextKnown.getBytes(StandardCharsets.UTF_8);
// number of known bytes of the key
int numberKeyBytes = 14; // 14 bytes = 112 bit known bits
// random key & iv
SecureRandom random = new SecureRandom();
byte[] key = new byte[16]; // 16 byte = 128 bit keylength
random.nextBytes(key);
byte[] iv = new byte[16];
random.nextBytes(iv);
System.out.println("key length: " + key.length + " data: " + bytesToHex(key));
// setup cbc
SecretKeySpec keySpec = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpec = new IvParameterSpec(iv);
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, keySpec, ivParameterSpec);
// get plaintext
byte[] plaintextByte = plaintext.getBytes(StandardCharsets.UTF_8);
byte[] ciphertextByte = cipher.doFinal(plaintextByte);
byte[] keyGuessed = new byte[16];
System.arraycopy(key, 0, keyGuessed, 0, numberKeyBytes ); // copy first 14 bytes = 112 bit
System.out.println("keyGuess length: " + keyGuessed.length + " data: " + bytesToHex(keyGuessed));
for (int a = 0; a < 256; a++) {
for (int b = 0; b < 256; b++) {
for (int c = 0; c < 256; c++) {
for (int d = 0; d < 256; d++) {
keyGuessed[15] = (byte) d;
//System.out.println("keyGuess length: " + keyGuessed.length + " data: " + bytesToHex(keyGuessed));
decryptAesCbc128(keyGuessed, iv, ciphertextByte, plaintextKnownByte);
}
keyGuessed[14] = (byte) c;
}
keyGuessed[13] = (byte) b;
}
keyGuessed[12] = (byte) a;
}
}
private static boolean decryptAesCbc128(byte[] key, byte[] iv, byte[] ciphertext, byte[] plaintextbyteKnown) throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException {
SecretKeySpec keySpecDecode = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecDecode = new IvParameterSpec(iv);
Cipher cipherDecrypt = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipherDecrypt.init(Cipher.DECRYPT_MODE, keySpecDecode, ivParameterSpecDecode);
byte[] decryptedtext = new byte[0];
try {
decryptedtext = cipherDecrypt.doFinal(ciphertext);
} catch (IllegalBlockSizeException e) {
e.printStackTrace();
} catch (BadPaddingException e) {
//e.printStackTrace();
return false;
}
// partial array comparison
boolean found = ByteBuffer.wrap(decryptedtext, 0, 5).equals(ByteBuffer.wrap(plaintextbyteKnown, 0, 5));
if (found == false) {return false;}
System.out.println("*** key found ***");
System.out.println("key length: " + key.length + " data: " + bytesToHex(key));
System.out.println("plaintext: " + new String(decryptedtext));
System.exit(0);
return true;
}
private static String bytesToHex(byte[] bytes) {
StringBuffer result = new StringBuffer();
for (byte b : bytes) result.append(Integer.toString((b & 0xff) + 0x100, 16).substring(1));
return result.toString();
}
}

How do I access fields in an active Revit schedule via RevitPythonShell/IronPython?

I'm working on an IronPython script for Revit 2016. For starters, I'm trying to access values (as text) in an active Revit schedule, and load them into a variable. This works well enough for non-calculated values.
However, some of my schedule fields are calculated. Here's a sample schedule (all values here are calculated):
Schedule Snippet
The Revit API shows 2 methods, called TableView.GetCalculatedValueName()and TableView.GetCalculatedValueText(), which I'd like to use, but don't seem to work as advertised.
doc = __revit__.ActiveUIDocument.Document
uidoc = __revit__.ActiveUIDocument
schedule = doc.ActiveView
tableData = schedule.GetTableData()
print(tableData)
tableName = schedule.GetCellText(SectionType.Header,0,0)
qty = schedule.GetCalculatedValueText(SectionType.Body,4,1)
calcValName = schedule.GetCalculatedValueName(SectionType.Body,4,1)
print(tableName)
print("Calculated Qty is: " + qty)
print("Calculated Value Name is: " + calcValName)
Running this code (in Revit) produces the following output:
88-06134-01
Calculated Qty is:
Calculated Value Name is:
I'd like to point out that using TableView.GetCellText() actually works on calculated values, but it's the GetCalculatedValueName() that I'd really like to make work here.

I have done the same thing but in c# for Revit 2019. I hope you will understand it.
You can access the values of schedule data without exporting. Firstly, get all the schedules and read the data cell by cell. Secondly, create dictionary and store data in form of key, value pairs. Now you can use the schedule data as you want. I have tried this in Revit 2019.
Here is the implementation.
public void getScheduleData(Document doc)
{
FilteredElementCollector collector = new FilteredElementCollector(doc);
IList<Element> collection = collector.OfClass(typeof(ViewSchedule)).ToElements();
foreach (Element e in collection)
{
ViewSchedule viewSchedule = e as ViewSchedule;
TableData table = viewSchedule.GetTableData();
TableSectionData section = table.GetSectionData(SectionType.Body);
int nRows = section.NumberOfRows;
int nColumns = section.NumberOfColumns;
if (nRows > 1)
{
List<List<string>> scheduleData = new List<List<string>>();
for (int i = 0; i < nRows; i++)
{
List<string> rowData = new List<string>();
for (int j = 0; j < nColumns; j++)
{
rowData.Add(viewSchedule.GetCellText(SectionType.Body, i, j));
}
scheduleData.Add(rowData);
}
List<string> columnData = scheduleData[0];
scheduleData.RemoveAt(0);
DataMapping(columnData, scheduleData);
}
}
}
public static void DataMapping(List<string> keyData, List<List<string>>valueData)
{
List<Dictionary<string, string>> items= new List<Dictionary<string, string>>();
string prompt = "Key/Value";
prompt += Environment.NewLine;
foreach (List<string> list in valueData)
{
for (int key=0, value =0 ; key< keyData.Count && value< list.Count; key++,value++)
{
Dictionary<string, string> newItem = new Dictionary<string, string>();
string k = keyData[key];
string v = list[value];
newItem.Add(k, v);
items.Add(newItem);
}
}
foreach (Dictionary<string, string> item in items)
{
foreach (KeyValuePair<string, string> kvp in item)
{
prompt += "Key: " + kvp.Key + ",Value: " + kvp.Value;
prompt += Environment.NewLine;
}
}
Autodesk.Revit.UI.TaskDialog.Show("Revit", prompt);
}