Could someone please help me out…
On trying cell 1.2 - Source Configuration and even with
a successful dbt debug command output, when I try the
cell “1.2.5. Load the connection configuration into the
notebook with the following cell”, it gives me the following
error
psycopg2.OperationalError: could not translate host name “<DATABASE_ENDPOINT>” to address: Name or service not
known.
profiles.yml (under .dbt directory and under scripts directory)
has the correct postgres endpoint entry…
Hello @Rajeev_Prabhakar
Looks like you have missed this step:
The <DATABASE_ENDPOINT> should be replaced with the value from CloudFormation. Don’t forget to save the changes made to the file. Moreover, you need to make sure that the exact file is copied to ~/.dbt/profiles.yml in step 1.2.3.
Hello Amir,
Although, at an initial glance, it does “appear” that I missed the step, but, I had checked (the profiles.yml) before posting this question, that’s why encountering this error was odd.
I’ll start the lab all over and may be it will work this time…
I retried and am running into the error again. Also, I did
confirm again copying profiles.yml to ~/.dbt/profiles.yml.
I suppose, I must be missing another step or doing
something incorrectly…
(lab-venv) voclabs:~/.dbt cat profiles.yml classicmodels_modeling: target: dev outputs: dev: type: postgres threads: 1 host: de-c4w1a1-rds.xxxxxxxxxxxxxxxxx.rds.amazonaws.com port: 5432 # optional dbname: postgres schema: classicmodels user: postgresuser password: xxxxxxx (lab-venv) voclabs:~/.dbt
Hello again @Rajeev_Prabhakar
I tried the lab right now and it seems to be working fine. Please, answer the following so that we dig deeper into the issue:
dbt debug in the cloud9 terminal? The result should be like this:
import yaml
with open("./scripts/profiles.yml", 'r') as stream:
data_loaded = yaml.safe_load(stream)
DBCONFIG["host"]
./scripts/profiles.yml file and outputs the the value in front of host variable. If you see <DATABASE_ENDPOINT>, it means that the changes has not been made to the file.
For step 1, the output I got earlier was
(lab-venv) voclabs:~/environment/classicmodels_modeling $ dbt debug
/home/ec2-user/environment/lab-venv/lib64/python3.9/site-packages/networkx/utils/backends.py:135: RuntimeWarning: networkx backend defined more than once: nx-loopback
backends.update(_get_backends(“networkx.backends”))
11:37:08 Running with dbt=1.8.1
11:37:08 dbt version: 1.8.1
11:37:08 python version: 3.9.20
11:37:08 python path: /home/ec2-user/environment/lab-venv/bin/python3
11:37:08 os info: Linux-6.1.119-129.201.amzn2023.x86_64-x86_64-with-glibc2.34
11:37:08 Using profiles dir at /home/ec2-user/.dbt
11:37:08 Using profiles.yml file at /home/ec2-user/.dbt/profiles.yml
11:37:08 Using dbt_project.yml file at /home/ec2-user/environment/classicmodels_modeling/dbt_project.yml
11:37:08 adapter type: postgres
11:37:08 adapter version: 1.8.1
11:37:08 Configuration:
11:37:08 profiles.yml file [OK found and valid]
11:37:08 dbt_project.yml file [OK found and valid]
11:37:08 Required dependencies:
11:37:08 - git [OK found]
11:37:08 Connection:
11:37:08 host: de-c4w1a1-rds.cdyoiwwa0br5.us-east-1.rds.amazonaws.com
11:37:08 port: 5432
11:37:08 user: postgresuser
11:37:08 database: postgres
11:37:08 schema: classicmodels
11:37:08 connect_timeout: 10
11:37:08 role: None
11:37:08 search_path: None
11:37:08 keepalives_idle: 0
11:37:08 sslmode: None
11:37:08 sslcert: None
11:37:08 sslkey: None
11:37:08 sslrootcert: None
11:37:08 application_name: dbt
11:37:08 retries: 1
11:37:08 Registered adapter: postgres=1.8.1
11:37:09 Connection test: [OK connection ok]
For step 2 you mentioned above, when I insert a cell above
and run it, get an output ‘<DATABASE_ENDPOINT>’
which is a problem, but, not sure how / why it is getting that incorrect value. b/c in my virtual env, profiles.yml under scripts
dir as well as under ~/.dbt directory has correct host entries
(lab-venv) voclabs:~/environment/scripts
-rwxr-xr-x. 1 ec2-user ec2-user 312 Dec 24 11:29 profiles.yml
(lab-venv) voclabs:~/environment/scripts cat profiles.yml
classicmodels_modeling:
target: dev
outputs:
dev:
type: postgres
threads: 1
host: de-c4w1a1-rds.cdyoiwwa0br5.us-east-1.rds.amazonaws.com
port: 5432 # optional
dbname: postgres
schema: classicmodels
user: postgresuser
password: adminpwrd
(lab-venv) voclabs:~/environment/scripts pwd
/home/ec2-user/environment/scripts
(lab-venv) voclabs:~/.dbt - cat profiles.yml
classicmodels_modeling:
target: dev
outputs:
dev:
type: postgres
threads: 1
host: de-c4w1a1-rds.cdyoiwwa0br5.us-east-1.rds.amazonaws.com
port: 5432 # optional
dbname: postgres
schema: classicmodels
user: postgresuser
password: adminpwrd
(lab-venv) voclabs:~/.dbt pwd
/home/ec2-user/.dbt
(lab-venv) voclabs:~/.dbt
Can you run this please?
cat ./scripts/profiles.yml
drwxr-xr-x. 10 ec2-user ec2-user 16384 Dec 24 11:24 classicmodels_modeling
drwxr-xr-x. 3 ec2-user ec2-user 138 Dec 24 11:29 scripts
-rw-r–r–. 1 ec2-user ec2-user 54380 Dec 24 13:09 C4_W1_Assignment.ipynb
(lab-venv) voclabs:~/environment $ cat ./scripts/profiles.yml
classicmodels_modeling:
target: dev
outputs:
dev:
type: postgres
threads: 1
host: de-c4w1a1-rds.cdyoiwwa0br5.us-east-1.rds.amazonaws.com
port: 5432 # optional
dbname: postgres
schema: classicmodels
user: postgresuser
password: adminpwrd
(lab-venv) voclabs:~/environment
Tue Dec 24 13:32:07 UTC 2024
@Rajeev_Prabhakar I’m actually at my wits’ end here. I don’t know how it’s possible that the notebook cell is getting a wrong host value from the file 
I suggest you change the line DBHOST = DBCONFIG["host"] in the notebook cell, and simply put the host value in front of it, like DBHOST ='de-c4w1a1-rds.cdyoiwwa0br5.us-east-1.rds.amazonaws.com'
sure, I’ll do the cell entry per your advice…thank you !