25
submitted 4 months ago by driving_crooner to c/python@programming.dev

Hi, I want to know what is the best way to keep the databases I use in different projects? I use a lot of CSVs that I need to prepare every time I'm working with them (I just copy paste the code from other projects) but would like to make some module that I can import and it have all the processes of the databases for example for this database I usually do columns = [(configuration of, my columns)], names = [names], dates = [list of columns dates], dtypes ={column: type},

then database_1 = pd.read_fwf(**kwargs), database_2 = pd.read_fwf(**kwargs), database_3 = pd.read_fwf(**kwargs)...

Then database = pd.concat([database_1...])

But I would like to have a module that I could import and have all my databases and configuration of ETL in it so I could just do something like 'database = my_module.dabase' to import the database, without all that process everytime.

Thanks for any help.

you are viewing a single comment's thread
view the rest of the comments
[-] gedhrel@lemmy.world 1 points 4 months ago

If things are changing a bit each month, then in your module rather than a plain variable assignment

darabase = ...

you might want a function that you can pass in parameters to represent the things that can change:

def database(dir, ...):
    ...
    return ...

Then you can call it like this:

from database import database
db = database("/some/path")

... gope that makes some sense.

this post was submitted on 25 Jun 2024
25 points (90.3% liked)

Python

6325 readers
236 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events

PastNovember 2023

October 2023

July 2023

August 2023

September 2023

🐍 Python project:
💓 Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
Feeds

founded 1 year ago
MODERATORS