2023-09-04 14:45
Question: Rather than using the AWS Interface - how can I grab the secret value of the AWS secret using AWS CLI ?
Answer:
Step 1: AWS Security required to access - secretsmanager:ListS… Read More
Blog Directory > Technology Blogs > dba-ninja.com technology Blog >
Dba-ninja.com Blog
dba-ninja.com
Tags:
backup
region
aws dms
dynamodb
aws backup
inline lob
sql server
encryption
secret
dms describereplicationinstances
An agile dba focused on data privacy,data protection and management of information and information systems, relational, non-relational, structured, non-structured, semi-structured, on-premises, in the cloud
2023-08-30 13:51
Question: I've read about the new switch in AWS Backup service api called copy-source-tags-to-restored-resource , and have successfully applied the tag during the restore procedu… Read More
2023-08-16 16:03
Independent AWS encryption refers to the process of encrypting using a AWS KMS (AWS Key Management Service) key associated with the AWS backup vault. Not all AWS DBMS types are s… Read More
2023-08-07 10:16
The AWS Database Migration Service (DMS) has methods for moving LOB columns from one database engine to another. The functionality to manage the LOBs is part of the Database Migration… Read More
2023-07-27 15:50
Some notes on creating a cross region RDS read replica & if necessary , to promote the Read Replica to a standalone RDS instance using AWS CLI
The purpose of these notes is t… Read More
2023-07-26 12:28
Question : We're trying to run a DMS migration task which is taking over 3.5 hrs - and would like to decresase the runtime. Are there some infrastructure or process changes changes we… Read More
2023-07-13 12:55
One of the problems when migrating from SQL Server to PostgreSQL is dealing with PostgreSQL requirement for every object name to be unique within a schema. The DBA could face thi… Read More
2023-07-06 15:16
Question: Are index names unique in Postgres? I'm attempting to create an index with the same name on two different tables and getting the error message:
ERROR: relation "idx_myindex" alread… Read More
2023-07-04 10:14
Question: In Region 1 The DynamoDB tables are being backed up to the MyBackupVault (Region1) And I can see the backups in the MyBackupVault (Region1)But if I check the matching&n&hell…Read More
2023-06-28 15:36
Using AWS Backup cross-region copy replication requires some extra details to complete the steps successfully/
The scope for this post is RDS . I will also add details about Aurora & Dyn… Read More
2023-06-20 11:15
When the AWS Backup Service takes over the RDS Automated Backups there are a number of modifications occuring - requiring some attention.
The integration of AWS Backup and the RDS serv… Read More
2023-06-16 08:27
I'm researching different Disaster Recovery & Availability option for AWS hosted databases, As part of this research - I've also been researching various methods on how an application wi… Read More
2023-06-02 10:49
DMS Context logging is only available in AWS DMS version 3.5.0 or higher
If your Replication Instance is not already at 3.5 , In Replication Instance Modify your Replication Instance to 3.5… Read More
2023-06-01 07:42
Terraform explained in 15 mins
A quick summary of the Terraform explained in 15 mins YouTube video . To get the full details watch this excellent video - linked below
What… Read More
2023-05-25 12:39
Part of setting up an AWS Database Migration Service (DMS) Migration Task includes createing Endpoints . The Endpoints are broken down into two types: Source & T… Read More
2023-05-15 07:12
Question : In an AWS DMS Migration task I'm trying to migrate a SQL Server table with columns defined as User Defined Data Types but getting a message with with Table… Read More
2023-05-12 15:26
Question: Quite often , after completing an AWS Database Migration Service (DMS )database migration task - I discover columns or tables missing from the target database. There can be many re… Read More
2023-05-05 10:42
Question: I'm trying to restore an RDS backup from AWS Backup but the original customer managed KMS key is no longer available. When I attempt the Restore I get the followi… Read More
2023-04-24 10:52
Question:
I'm trying to migrate some SQL Server tables with varbinary(MAX) data type defined columns using the AWS Database Migration Service (DMS). The target is AWS RDS P… Read More
2023-03-29 09:56
Question: How can I convert an existing column in Postgresql to become an auto generated column ? The existing table column is an integer and is a Primary Key . There are already… Read More
2023-03-15 16:54
Question: When attempting the following aws cli statement I get an error message
aws rds restore-db-instance-from-db-snapshot \
--db-instance-identifier mypostgresRDS \… Read More
2023-03-09 10:02
Question: Is Terraform declarative or imperative and is it suitable for DB restore procedures ? I'm working on developing procedures to integrate ad-hoc Restore procedures across… Read More
2023-03-06 09:41
Question: How to drop all tables from a PostgreSQL database schema?
Answer: If you have a requirement to drop all from a PostgreSQL database schema this code snippet will allow you to a) cr… Read More
2023-03-01 11:35
Question: What is the difference in an AWS option group and AWS parameter group?
Answer: The definitions of Parameter Groups and Option Groups are as follows
&nbs&hell…Read More
2023-02-02 16:53
In a large organization maintaining a separated AWS account for every line of business or application can be a complicated structure. Here's a list of lessons learnt and factors to consider… Read More
2023-02-02 11:38
Question: Since AWS SCT Build 670 - it's not possible to apply the Windows Authentication Library through the Global Settings tab. How can I access the Authentication Library ?
Answser: Ther… Read More
2023-01-30 10:03
Question: we're researching using the AWS Database Migration Service (DMS) to migrate a homogenous migration between on-premises PostgreSQL to Aurora PostgreSQL.
Within the database th… Read More
2023-01-24 10:51
Question: After an upgrade from MongoDB 4.4 to MongoDB 5 some start up warning signs have appeared .
{"t":{"$date":"2023-01-20T13:35:05.354+00:00"},"s":"I", "c":"REPL", "id":5569202, "ctx":… Read More
2023-01-13 09:41
Question : When attempting to create an new RDS instance I get the message - RDS Client: DB Instance quota exceeded. That does sound self-explanatory - i.e the accoun… Read More
2022-12-28 16:09
RDS - AWS Backup service supports RDS continuous backups. When you configure the Backup Plan change the "EnableContinuousBackup" variable to True.
After creating the… Read More
2022-12-15 14:34
DynamoDB supports On-Demand backups and Point-In-Time backups
Notes on DynamoDB backups
1) When PITR is enabled , DynamoDB maintains continuous backups and can restore to any point in… Read More
2022-12-14 13:06
Deciding storage type & instance type for a specific database workload can be a complicated task . One of the biggest challenges facing DBAs is the ability to utilize the AWS resources i… Read More
2022-12-05 13:35
Up until now the AWS Schema Conversion Tool (SCT) and the Database Migration Service (DMS) have remained as separate installations. The SCT downloaded and installed on a local desktop or ser… Read More
2022-12-01 03:22
These are notes from AWS re:Invent 2022. They are in no particular order but are points I noted as I talked\listened to various Engineers and Architects
Finops from the start of any pl… Read More
2022-11-16 10:43
Question : When I use the AWS CLI --output table option , I get characters returning which is breaking the output . Here is an example:
aws rds describe-db-snapshots --db-i… Read More
2022-11-14 15:55
Question: I want to create security group for my AWS Instance . The idea is to associate the security group with an RDS Instance , and attach some ingress rules bound to a port
Answer… Read More
2022-11-07 11:28
This is a step by step guide on how to use AWS CLI and restore an RDS instance backed up with the AWS Backup Service . The guide will take you through the IAM role set up , retrieve the reco… Read More
2022-10-12 06:59
Serverless is a term related to Cloud Computing. The primary objective of the Serverless model is for the cloud provider to allocate and manage the server resources for the customer. Althoug… Read More
2022-10-10 12:40
Question: I'm dealing with a SQL Server to PostgreSQL conversion using the AWS Schema Conversion Tool Assessment Report . One of the responses in the Assessment Report is :
Issue 7681… Read More
2022-09-27 15:26
Question: I have to prepare a report to assess the potential workload of converting SQL Server databases to AWS PostgreSQL targets. There are a large number of databases , so i'd like to ide… Read More
2022-09-15 14:21
AWS CLI commands and queries I use on a regular basis
--get the aws cli help pages
aws help
RDS
--list RDS instances
aws rds describe-db-instances
--list… Read More
2022-09-01 15:38
Migrating on-premises SQL Server to Aurora PostgreSQL using AWS Database Migration Services (DMS) can be complex. This is a list of best practices which I've learnt over a number of mi… Read More
2022-08-30 09:27
The Encryption algorithm , Format Preserving Encryption (FPE) maintains the format of the data. For example , if you had a credit card with 16 digits - 01234567890123456 - … Read More
2022-08-18 08:30
Question: I executed the net use statement and got a System error 1219 has occurred
net use * \\myserevr\c$ /u:mydomain\myid mypasswordSystem error 1219 has occurred
Multi… Read More
2022-08-03 15:22
The Babelfish Compass Tool is an assessment tool for compatibility between SQL Server & Babelfish.
The following is an outline of the migration process with Babelfish… Read More
2022-07-25 15:29
A common question arising from a conversation about AWS Database Migration Service (AWS DMS) is the rollback strategy. AWS does a good job at documenting & discussing solutions for on-pr… Read More
2022-07-22 14:26
Question: I'm currently using MongoDB on-premises and reviewing alternative options for storing NoSQL data. Our usage of MongoDB is very limited from a development pov , and looking for a mu… Read More
2022-07-14 16:05
A question that comes up often when planning moving from an on-premises DBMS utilising Active Directory as an identity access management system to a Cloud Native setup - is what… Read More
2022-07-13 12:08
dblink is a Postgresql extension and supports executing a query in a remote database, dblink which is very similar to Linked Servers in SQL Server.
There is a substantial s… Read More
2022-07-12 07:32
PostgreSQL supports SSL connections. PostgreSQL has a concept of SSL modes , each mode serves a different objective . Starting from no encryption - sslmode = Disable , right through to… Read More
2022-07-08 12:30
Is AWS Backup Service a suitable replacement for the automated RDS service backups? As more services are deployed - it makes sense to think about how to employ an approach which includ… Read More
2022-07-07 10:12
What does AWS recommend when you enable encryption for your RDS PostgreSQL encryption and Aurora PostgreSQL?
Most orgs are interested in 3 general categories of encryption for database… Read More
2022-06-29 09:06
I received a question from a XAMPP developer about how to find the data directory in MariaDB. They were receiving the error : ERROR 1010 (HY000): Error dropping database (can't rmdir './myda… Read More
2022-06-28 15:39
I watched a good video on AWS DMS - a couple of years old - but good fundamentals . Suitable as an introduction - explaining use cases & reasons to use AWS DMS
AWS re:Invent 2021 - Dive… Read More
2022-06-27 13:35
The main tables for accessing mysql backup history is : mysql.backup_progress & mysql.backup_history , when the mysqlbackup command is used
Use the backup_… Read More
2022-06-23 09:51
Our Privacy Policy was last updated on 23 June 2022.
This Privacy Policy describes Our policies and procedures on the collection, use and disclosure of Your information when You use the Serv… Read More
2022-06-17 15:55
A question that comes up is how to validate data migrated between two databases using the AWS Data Migration Service (DMS) .
AWS DMS supports a validation process as part… Read More
2022-06-16 09:30
Question: When attempting to create a PostgreSQL table with the CREATE TABLE command I get an error message:
ERROR: permission denied for schema northland_claims LINE 1: create table mytest(… Read More
2022-06-15 15:30
The choice between Amazon RDS for PostgreSQL & Amazon Aurora PostgreSQL is dependant on the a number of factors : features required, if the db is migrating from one of… Read More
2022-06-14 13:17
Question: How can I create a PostgreSQL user for CRUD operations in a specific database?
Answer: The CRUD operations need to cover CREATE , READ ,UPDATE and DELETE. This i… Read More
2022-06-10 16:22
There are different types of encryption supported by PostgreSQL. If the requirement is to support encryption for specific fields - look into the pgcrypto module
create table cry… Read More
2022-05-27 10:10
This query will report on connections and ssl status. The pg_stat_ssl view reports on connection SSL . Adding the join to pg_stat_activity gives some extra connection… Read More
2022-05-23 13:06
Question: Which should I use ?PostgreSQL drivers for Unicode v ANSI ? These options appear when I attempt to configure am ODBC DSN for a connection to a PostgreSQL da… Read More
2022-05-17 09:43
Question: I know that “public” is PostgreSQL default scheme. I want to change the default schema name , as there is a procedure to migrate tables into a dedicated schema … Read More
2022-05-13 11:01
Question: How can I list the Constraints on a PostgreSQL table ?
Answer: This script will list PostgreSQL constraints . It achieves the results via the pg_catalog.pg_constraint&n&hell…Read More
2022-05-10 10:57
Question : I'm trying to execute a Python script but getting an Import Requests module error . How can I fix ?
Answer: The Python Requests module does not come with the default Python instal… Read More
2022-04-28 08:12
Question: I'm executing a DROP DATABASE my_db and getting this message , how can I kill user session connections for this specific database ?
DETAIL: There is 1 other session using the datab… Read More
2022-04-27 14:21
Question: I have some tables on database on a PostgreSQL server instance. I want to copy the schema and data to another PostgreSQL server instance . Is there a straightforward process… Read More
2022-04-20 10:21
This is slightly off-piste , but I've recently reignited my sci-fi reading , and decided to list out my favourite sci-fi novels . No particular order - I'll add more. Feel free to add your o… Read More
2022-04-14 09:38
Question: I tried to connect to a MongoDB server just using the mongodb prefix - mongo "mongodb://myuser:mypassword@myurl/test?retryWrites=true&w=majority" and got an error message… Read More
2022-04-11 08:57
Question: I've migrated a SQL Server database to PostgreSQL , but the table names and column names - from the original SQL Server database were a mixture of upper case and lower case. The ta… Read More
2022-04-07 15:40
There are a number of challenges to migrating MS Access tables to PostgreSQL. Here is a simple step by step that I used to successfully migrate a series of tables and data… Read More
2022-03-24 10:12
Watched a good quality talk on Amazon Aurora by Richard Waymire. Added some notes below - and some snapshots of the video.
There is a link to the full video at bottom of post&nbs&hell…Read More
2022-03-18 16:34
Babelfish for Aurora PostgreSQL supports a TDS port and a postgreSQL port.
TDS port use a sql server client e.g SSMS , sqlcmd
postgeSQL port use a postgreSQL client e.g pgAdmin… Read More
2022-03-16 16:30
- Babelfish for Aurora PotgreSQL : Single -Mode versus Multiple - mode ? This will impact how many user database can co-exist on a single Babelfish Instance .
To illustrate… Read More
2022-03-04 09:13
Playing Wordle every morning is one of my satisfying pleasures. The fantasy is to crack the Wordle enigma first try - but in reality that would be blind luck. … Read More
2022-02-21 13:07
Question: How can I edit the Global settings driver information on the AWS Schema Conversion Tool via a configuration file ? Rather than using the GUI interface ?
When I l… Read More
2022-02-04 10:44
-Translation Service between SQL Server & Aurora PostgreSQL, more specifically : Amazon Aurora PostgreSQL-Compatible Edition that enables Aurora to understand commands from applications… Read More
2022-01-26 13:42
Depending on the complexity and feature usage within SQL Server - a migration or conversion to PostgreSQL can be a challenge
Geographic data
PostgreSQL does not have geographic… Read More
2022-01-23 17:28
Question: I can print out the os environment variables using Python with :
import… Read More
2022-01-15 12:27
Question: How do I hide user input for a password entry using Python? So instead of the password displaying as the user types the password , the screen wouldn't change.
Ans… Read More
2022-01-05 16:45
Since I started working with databases there have been multiple paradigm shifts in management of data.
Some notable changes include
> Moving to different data types , e.g image… Read More
2021-10-01 16:26
Question: There is a cronjob backing up MariaDB databases through the mariabackup utility.
mariabackup --defaults-file=/my_dir/.backup-my.cnf --backup --parallel=$PARALLEL --no-timesta… Read More
2021-09-20 15:52
The isolation levels define which database objects are locked and how long that lock is held for. They are one of the critical concepts related to concurrency in transactions.There is… Read More
2021-09-08 10:00
Question: I've reached some drive storage limits on SQL Server supporting large modelling databases. There is plenty of underlying storage available but the current disk is initialized with… Read More
2021-09-02 08:42
You're constantly programming, analysing,learning,troubleshooting , and developing some great experience ...and there was probably a time ehrn you suddenly realized you're a PRO.
There isn't… Read More
2021-08-25 14:14
Embrace the CHAOS! (aka resiliency)
Chaos engineering is defined as "the discipline of experimenting on a distributed system in order to build confidence in the system’s ca… Read More
2021-08-24 15:58
If you're interested in a wider perspective on managing server systems, i've just read a comprehensive book , which offers a framework .
Building Secure & Reliable Systems by Heath… Read More
2021-08-16 16:02
Question: I have some directory backups available to me created using the following command line:
mysqlbackup --defaults-file=/myfolderpath/.backup-my.cnf --backup-dir=$BKUP_DIR… Read More
2021-07-22 09:32
Question: We're reviewing extending an application into a cloud provider. Part of the review requires analysing consequences of using one-way trust or two- way trust. Could you explain in si… Read More
2021-07-19 13:36
Database Reliability Enginnering
Key principle: Database engineering and systems engineering can work together
Data engineering and SRE , i.e applying SRE to database engin… Read More
2021-07-08 10:28
Question: We have event log data going into a MongoDB collection. There is no requirement to keep this data beyond 60 days. Is there a way of automating the process to delete the rows , with… Read More
2021-06-28 13:14
What is the process for a server selecting the Active Directory domain controller? When a server requires a Domain Controller , what are the steps leading to a Domain Controller decision. I'… Read More
2021-06-24 09:03
AWS Database Migration Service (AWS DMS) is a cloud service supporting relational databases, data warehouses, NoSQL & other various datastores migrations.
Two main use cases… Read More
2021-06-23 15:43
Question: How can I get the subnets of a server's Active Directory site?
Answer: There are a few different methods, typically it can be a complex issue to get the subnets of the Activ… Read More
2021-06-15 16:04
Question: We've started managing a monitoring system with an API interface - generating output from a MongoDB database. We have no knowledge of MongoDB and just need a quick primer on… Read More
2021-06-09 10:53
Question: I'm getting an error message on the application connecting to a PostgreSQL database Caused by: org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserv… Read More
2021-05-18 13:49
Question: I jump around on various MongoDB databases and want to create a process to programatically check the CURRENT USER. What is the MongoDB command to check the current user… Read More
2021-05-17 15:25
Question: How do I create an index on a MongoDB ? I'd like to create it as a background activity - which will avoid disabling the database activity
Answer: Find a sequence of statemen… Read More
2021-05-12 11:36
In this age of Ransomware , backups and protection of backups is critical. One question asked in any review of database protection against attacks is whether the backups are immutable?
Defin… Read More
Subscribe to Dba-ninja.com
Get updates delivered right to your inbox!