RSS

Tag Archives: sql

A COVID19 Story with Azure Databricks

A COVID19 Story with Azure Databricks

Before I get to my story, I would like to say that you can download the notebooks I created and ran in Azure Databricks here: https://github.com/thesqlpro/blog/tree/master/notebooks

The source of my data was: https://covidtracking.com/data

I chose not source my data directly from Maryland’s State Government site because the format was not easy to use. The official Maryland Government provided data basically has each day as a column and had the rows as Zip Codes — not as easy as the data provided from the site above. So there may be few discrepancies between the data on a day to day basis, but the totals are identical. You can read about their methodologies of retrieving data from various official State Government websites and the quality of each.

This post is in no way intended to attack anyone, be part of a political movement, promote any agendas political/financial/social, or support any causes out there except one: highlight how data and statistics can be used to tell stories. As data professionals, we need to give importance to the quality of data and the quality of data reporting. Basically this is a lesson in data visualization and telling stories with data.

Read the rest of this entry »
 
1 Comment

Posted by on June 3, 2020 in Other

 

Tags: , , , , , , , ,

Introduction to Azure Databricks Presentation – Video Recording

Introduction to Azure Databricks Presentation – Video Recording

I presented an introduction to Azure Databricks on May 22, 2020 to one of our local SQL Server User Groups here in the Washington DC area. It was done online due to the Covid19 restrictions on gatherings. The good that came out of doing it online was that it was recorded and published on YouTube. Here is the video, I hope you enjoy learning!

Code can be found at my new GitHub Repo:

https://github.com/thesqlpro/blog

 
Leave a comment

Posted by on May 28, 2020 in Other

 

Tags: , , , , , , , ,

SQL Server Management Studio Rainbow Edition

SQL Server Management Studio Rainbow Edition

Ever wanted to know the reason behind the Database color scheme in SQL Server Management Studio?

I have always wondered why all the Databases in management studio were Yellow (Online), Grey (Read-Only), Lighter Yellow with text next to it (something like Restoring), or Yellow with a Red Icon (Offline). Have you ever wondered why there aren’t any more useful colors like Red for when a Transaction Log is Full or Purple when your Statistics are Outdated. I have a theory on this that I have entitled “The Theory that you have to actually work to call yourself a DBA.”

colorful databases

Read the rest of this entry »

 

Tags: , , , , , , , ,

How to Avoid a Database “Heart Attack”

How to Avoid a Database “Heart Attack”

For the past couple of months I have been working with Ben Weiss a Digital Marketing Strategist at Infusive Solutions (http://www.infusivesolutions.com/). Our partnership was based on a mutual understanding that there is much needed career growth and guidance among IT Professionals. Since my background is primarily with Database Management Systems I have been helping him with writing articles, guides, and even interview questions for top professionals in the field, all for the purpose of furthering the careers of IT professionals.

Our latest project is a Lessons Learned pamphlet based on several articles I wrote about a year ago after attending open heart surgery (as an observer, not a patient). Here is the link to the final product, I hope you enjoy and benefit from it.

Here is a link to a summary of the project
http://www.infusivesolutions.com/blog/bid/98928/3-Ways-Business-Observation-Catalyzes-SQL-Server-DBA-Job-Growth

 

Tags: , , , , , , , ,

Email Response: SQL Server 2012 Planning and Architecture

Email Response: SQL Server 2012 Planning and Architecture

I received an email from a connection on LinkedIn seeking some advice for architecture on a new reporting system. Although I don’t know the gentleman personally, he is a SQL Server professional and I felt obliged to help out a fellow colleague in the field. The email is included below and my suggested solutions following it.

Read the rest of this entry »

 

Tags: , , , , , ,

Querying the Management Data Warehouse for Disk Usage over time (Data Collector)

Querying the Management Data Warehouse for Disk Usage over time (Data Collector)

A part of every DBA’s job is to justify all those disk space requests. The phrases “I just need it” or “Trust me this database is going to grow one terabyte this year” might not convince your boss.

Many people are under the impression that disk space is cheap since they base that notion off of the shell shocker they saw on NewEgg or the weekly circular from Best Buy. If you are using consumer SATA drives from retail outlets for your production databases please do the following:
1. Update your resume
2. Apply for jobs at least 1000 miles from your current location
3. Hope the next DBA that replaces you has more common sense

We’re in the times of SANs, iSCSI, SAS drives, SSD drives and RAID architecture! You should be forking out almost the cost of your server in disk space. Unless of course losing data is an option where you work.

Now that I’ve bored you or intrigued you (or something in between) I can get to my point. There is a wonderful tool that comes with SQL Server 2008 and later called the Management Data Warehouse (MDW) also known as the Data Collector.

Read the rest of this entry »

 

Tags: , , , , ,

SQL Server 2012 New Feature – The Sequence Object

SQL Server 2012 New Feature – The Sequence Object

One of the cool new features with SQL 2012 is the Sequence Object.  One of the worst parts of this feature is that whenever I mention it’s a cool new feature a lot of people say “that has been in Oracle forever.” Patience is a virtue!!

Anyway, so I like to consider the Sequence Object as a “Global Identity” that can be shared by many objects in the same database, or even across databases on the same server (I have not tried it via Linked Servers because I only have one test SQL 2012 server). You can take values from the Sequence Object and insert them into tables or you can have a column in a table default to a value from the Sequence Object. A warning to the wise: If you assign a default value for the table as a Sequence Object then you cannot drop it because of dependencies. That is actually a pretty good security measure.

I have included a code snippet so that you can play around with this new feature and I’ve put a few screen shots to show how you can manage it via the GUI. Make sure you run the code section by section; I have included a lot of comments to let you know what to expect while running it.

Read the rest of this entry »

 

Tags: , , ,

Open Heart Surgery Lessons Learned for IT Part 3 – Monitoring Health and Disaster Recovery

Open Heart Surgery Lessons Learned for IT Part 3 – Monitoring Health and Disaster Recovery

One of the biggest issues we have in health is poor monitoring.  This is also the case in health of many of our systems.  As a DBA, I’m always concerned about the health of my database servers; mainly because I would like to keep my job and not get the 3am call that something has gone terribly wrong. However, with that said, no matter how careful we are there is always the probability of something bad going wrong.

This is part three of my lessons learned and will have more of a focus on Database monitoring and Disaster recovery.

Read the rest of this entry »

 

Tags: , , ,

Five Seconds Transaction Log Shrink


This is somewhat of a quick or lazy way to truncate and shrink the transaction log because all you have to do is replace the database name and change your size requirements. It’s not the cleanest or best approach I admit, but sometimes I find myself on a development or staging server and I need some space quickly. There is no requirement on the server to preserve historic data, and since it is not highly transactional I don’t care what is on the log file to begin with. This happens often when running data aggregations in our DataMart environment where log files can grow up to 100GB+ with staging data that is not useful after the process is complete.

Read the rest of this entry »

 

Tags: , , , ,

 
%d bloggers like this: