While working on a new feature, you find some small other things to fix. Since these changes are unrelated, you probably shouldn’t commit them together. You could revert these changes to redo them separately, but who wants to do extra work? Fortunately, you can now select which chunks or even lines of changes to add to your commit. You can commit the rest separately or even move it to a new change list.
In this tutorial, we’ll look at exploring project structure with IntelliJ IDEA‘s Dependency Matrix.
When working with large, complex software projects, we need to understand the dependencies between components in your projects. IntelliJ IDEA has a feature called Dependency Structure Matrix (DSM), or Dependency Matrix, that can help us with this.
When you are new to a project, the Dependency Matrix can help you get an overview of the dependencies in the project. The Dependency Matrix can also help us get an idea of how hard it will be to split a project, based on the dependencies between components. Finally, it can help us find and untangle cyclic dependencies.
It offers a matrix of the components in our project to help you and highlights the usage flow between them. Let’s take a look!
Opening the Dependency Matrix
You can open the Dependency Matrix from the main menu by going to Code | Analyze Code | Dependency Matrix.
Alternatively, use Find Action (⌘⇧A on macOS, or Ctrl+Shift+A on Windows/Linux) and search for “matrix”.
To use this feature, the Dependency Matrix plugin needs to be enabled.
When opening the Dependency Matrix, specify the scope you want to analyze. You can select the Whole project or specify a Custom scope, and whether to include test sources or not.
After defining the scope, click Analyze.
If the project’s class files are out of date, the analysis may result in incomplete or incorrect data. To avoid this, IntelliJ IDEA will prompt you to compile the project before continuing the DSM analysis. Click Yes to build the project and make sure everything is up-to-date.
The DSM tool window will open in a popup, showing a matrix of your project’s components.
Interpreting the Dependency Matrix
Let’s take a look at how to interpret the Dependency Matrix.
The row headers represent the program structure. In this example, the matrix contains the same modules from the Project tool window as rows.
Notice that the modules are not sorted alphabetically, as they are in the Project tool window. The matrix moves the components that are used the most to the bottom. This means that the components located at the top of the matrix depend on the components below.
Since this is a matrix, the column headers are the same as the row headers. They are omitted to save space. The dashes on the diagonal correspond to self-dependencies, which are not shown.
As you can see in the legend at the top right of the DSM tool window:
dependencies are shown in blue
mutual or cyclic dependencies, meaning that two components depend on each other, are shown in red
dependencies flow from green to yellow.
Dependencies
Dependencies are shown in blue. The numbers in the cells show the number of dependencies of the selected row on the selected column. An ellipsis (the three dots) in a cell means that there are more than 99 dependencies. Hover over the cell to get more information. In the example below, we see the tooltip “cucumber-java -> cucumber-core (209)”. This means that in this project the component cucumber-java (represented in the column) depends on the component cucumber-core 209 times.
You can click rows or cells to examine the relationship between the components in more detail.
When you select a row to see the relationship between the selected component and others, the selected row and corresponding column are highlighted to visualize row dependencies.
The column shows the dependencies of the selected row.
The row shows the dependencies on the selected row.
Remember from the legend that dependencies flow from green to yellow.
In this example, the core module is selected. You can see that this module uses several other modules, marked in yellow. In turn, you see that this module is used by several modules marked in green.
You can select different rows to see which components they use or are used by. You’ll see that the components at the top mostly use the components at the bottom, while the components at the bottom are mostly used by components at the top and no longer using other components themselves.
You can drill down further into specific cells. When you click a cell, one component will be marked green and the other will be marked yellow. The green component uses the yellow component. The corresponding cell (marked purple) will show dependencies in the other direction, in this case 0.
We can drill down even further.
Expand components
Right now everything is collapsed and only the modules are shown. You can click a module to expand its packages. The module name is shown to the left, the packages are shown as rows and the dependencies between the packages are shown in the matrix inside the box marked with a black line.
You can expand the packages further to show the classes in that package, and see the dependencies between classes inside the packages. You can expand modules and packages by clicking the arrows in the rows on the left, and collapse them again.
You can also expand the modules and packages by double-clicking a cell. We can collapse everything again by using the Flatten Packages button on the top left.
Limit scope
You can limit the scope of your Dependency Matrix. Right-click the row you want to look at in more detail, and from the context menu, select Limit Scope To Selection.
The limited scope will be opened in a new tab in the DSM tool window.
You can limit the view to see only selected dependencies. In the DSM tool window, right-click the cell representing the dependency you’re interested in and select Explore Dependencies Between.
The classes that produce these dependencies will be opened in a new tab in the DSM tool window. In contrast to the Limit Scope option, only classes which produce selected dependencies are left.
Another way to open the Dependency Matrix, with the option to select a specific scope, is from the Project tool window (⌘1 on macOS, or Alt+1 on Windows/Linux). Right-click an item in the Project tool window and select Analyze | Analyze Dependency Matrix.
Notice how you can now select the module or directory as scope for the Dependency Matrix, in addition to the whole project or a custom scope.
Navigate to relevant code
We can also navigate to the relevant code from the Dependency Matrix. To select a specific dependency for further source-code analysis, right-click the dependency you are interested in in the DSM tool window, and select Find Usages for Dependencies.
The Find tool window will open, showing the usages of the selected dependency. Close the DSM tool window to look at the results and explore the code you’re interested in. We can open the relevant code by double-clicking it in the Find tool window (⌘3 on macOS, or Alt+3 on Windows/Linux).
You can reopen the window again from the main menu by going to View | Tool Windows | DSM.
And remember that all windows can be opened from Recent Files (⌘E on macOS, or Ctrl+E on Windows/Linux) as well.
Cyclic dependencies
Remember from the legend that mutual or cyclic dependencies are shown in red. This means that two components depend on each other.
In a large application with multiple cyclic dependencies, you don’t need to expand all the nodes one by one to find all the cyclic dependencies. You can press F2 or select Go to Next Cycle from the context menu to quickly jump to the next cycle.
Summary
In this tutorial you’ve seen how the Dependency Matrix can help visualize and explore dependencies between components in your project.
IntelliJ IDEA Shortcuts Used
Here are the IntelliJ IDEA shortcuts that we used.
It’s a Friday, late in the afternoon. To end your work week in a clean way, you decide to get rid of some test data and files from your PC. You hit the enter button to drop a table from your local test database. Within a split second, you realize your error. Your body turns hot and cold at the same time. You double-check, but you already know the truth. You were connected to the production database and just deleted the table with all the customers…
Congratulations! You will never forget this day. It’s the day you become a real developer, and all senior developers will welcome you into their world, as they all have made a similar mistake at least once.
Database Disasters
Dropping a table or a complete database is a mistake that can happen very quickly. Another one is forgetting the WHERE part of your SQL statement. There is a big difference between DELETE FROM clients and DELETE FROM clients WHERE id = 1…
But the main mistake happens when you have multiple connections defined in your database UI and assume you are modifying the test database while having the production database open. And that’s a problem that actually should not be possible to happen, or at least not easily. Do developers need access to the production database? While it can be helpful to be able to check data directly in the live system, read-only rights are probably sufficient for most cases…
Another database disaster waiting to happen is when you forget to validate the input. SQL injection should be a well-known problem by now, but still, a lot of errors happen in this field, causing not only disasters but also security nightmares. Or, as XKCD nicely illustrates it:
Dates and Times
Working with dates and times is the real test that distinguishes beginners from experienced developers. The defining factor is not only their ability to handle them correctly but also the number of “yes, I have seen this problem before” moments.
Before we dive into the problems, let’s summarize the most important guideline when working with dates and times in your database: store them completely, including the timezone. This is an article that goes into more details about timestamptz in PostgreSQL.
Here are some interesting situations related to dates, times, and time zones.
Young developers will probably not remember what the Y2K problem was all about, but let’s just mention here that storing a year’s value as only two numbers is a bad idea. No, the year 2000 (“00”) was not before the year 1999 (“99”). And we may be facing a similar problem in the year 2038, as I wrote in an earlier blog: “Schedule your holiday for 2038“. While writing that article, I never imagined seeing an error being caused by that Y2K38 problem already in 2023!
Frank
Of course, if your company only operates in one time zone, you might think that you will escape the problem of having to deal with time zones. And this might be true… Until you move to the cloud. Now you’ll have to deal with time zones too! And there are many reasons why dates and time zones are fertile ground for great conference talks and famous blog posts.
Marit
Computer System Breakdowns
But be aware, we can not only make mistakes in our code and database! Our whole computer infrastructure is prone to errors!
Many years ago, I was working on my first big multimedia project for a light fixtures manufacturer to bring their expensive thick catalogs to CD-ROM (for young people, a blinking disk that could contain a whopping 640 MB of data…). We had a strict deadline and had a first working version after three weeks of hard work. But then disaster struck! The hard disk of my fancy blue iMac broke.
After many hours of investigation, I ended up with four conclusions:
The backup tapes on our server that ran daily to ensure we would never lose any work contained not a single file!
It would take weeks and a lot of money to send the hard disk to a data recovery company without any guarantee to be able to recover anything.
Backups must be checked regularly and you must try from time to time if you can recover a deleted file to make sure both the backup and restore process work OK.
I had to start my work again from scratch…
Another lesson I learned from that disaster: weeks of work on a project that you have never done before can be repeated in a few days as you learned a lot during those weeks, and now know how to do things. And in the end, it even gets a cleaner and better result!
Another system disaster: never, never, NEVER, type the command rm -rf /. The remove command parameter f removes all prompts, so you let it go ahead without asking for any confirmations. The r lets the remove command work recursively, meaning it will go through all nested directories. Combine this with /, being the very root of your hard disk, and you are heading towards a total nightmare…
Frank
Testing Mistakes
In the same category of testing versus production databases: mailing lists! The number of stories of test emails reaching thousands of clients is incredibly long.It’s still amazing how many times one still receives an email with all recipients in the TO field. Any programmed emailing system should generate a single mail per person. And if you really, really need to send to multiple persons with one message, the BCC field is your only friend!Another well-known example is sending test messages in production, like the time Airbnb sent test notifications to users around the world or an intern at HBO Max sent an integration test email to subscribers.
I can remember a story, in the early days of the internet, when networks were not that fast, and servers were smaller. Most companies had their own internal email server. But when someone mailed a big PDF report or other file to everyone at the company, the complete system could crash. Suddenly too much storage was needed, and a ping-pong of email error replies, led to a total breakdown of the server.
Frank
Similarly, I remember several situations where we’ve had to (re)upload batches of data and somehow miscalculated the size and processing time, leading to queues being full and data processing to be severely delayed.
Marit
Conclusions
While some of the mistakes mentioned here are honest mistakes made by developers, some of the bad events described here actually reveal a problem within the organization. Do developers need unrestricted access to the production database? Why are they able to access the entire production mailing list for a test? Some data (especially personal data) should be behind heavily closed doors with minimal access. But in many companies, all this data is widely accessible to the whole developer team (or worse). If this is not guarded by the organization, developers should always be professional and handle data with care.
In addition, tooling should be used in such a way as to make mistakes like these harder. For example, environments should be clearly marked so it’s easy to see whether you are working on a test environment or the production environment. In some cases, additional steps have to be taken in order to get access to a production environment. Evaluate potential risks in your organization and act accordingly.
But as much as you try to avoid them, mistakes do happen. In the end, it’s essential to keep in mind that people will mostly remember how you reacted to a disaster. After the initial adrenaline spike has settled down, grab a cup of coffee, cancel your plans for the following days, and fix the problem. Everything will be OK soon. Afterward, you will wear the badge of “Senior Developer” with pride…
When working with large, complex software projects, we need to understand the dependencies between components in your projects.
#IntelliJIDEA‘s Dependency Structure Matrix (DSM), or Dependency Matrix, that can help us with this! Let’s take a look at how to use the Dependency Structure Matrix to see dependencies between different components, like modules, packages, and classes. See how to identify dependencies, find cyclic or mutual dependencies, and visualize the flow of dependencies to see which components use or depend on other components and vice versa.
