Atlassian And New SSL Certificates

Or How CAs Drain Your Lifetime

When you check my activity, you simply can see, that the past weeks my time for writing was quite limited. Since I had a few very urgent projects, I had no time to care about my blog. But when it can get no worse, your website provider, that previously relied on the Symantec CA, decides to switch it’s root CA to some better and quite new CA that 1. is not listed in some JRA distributions‘ keystore and 2. uses trust chains with intermediate certificates. And this leads to an ugly situation when you run e.g. some Atlassian tool environment where the tools in turn use SSL to connect to each other. I would have not realized the problem so early, nor it would not have been so urgent, when I would not have decided to do all authentication (Bitbucket and Confluence) through JIRA’s Crowd-API.

But let’s start from the beginning… My provider, where I get my SSL certificate from (namely the german 1&1) used the Symantec CA for years. This CA somehow attracted Google’s anger, so Google decided to remove Symantec from the trusted CAs list of it’s chrome browser and announced this around beginning of 2018. 1&1 did not find a reason to get in hurry, so they kept their CA until mid of 2018. Then they started to remind their customers to update their SSL certificates, forcing their customers to hurry a lot. I realized, that there is some neccessity to follow their appeal, but I also felt to still have some time and do it, when the certificate expires…

Then the time came and my Chrome browser refused to show my Atlassian pages. So I logged in to my 1&1 Account and my Linux machine where JIRA & Co. runs, ordered a new SSL certificate, copy-pasted it into my Nginx configururation (which I use as an SSL proxy) and everything went fine at the first glance. I could log in to JIRA without a hassle, did not see any JIRA warnings or hovers and my browser also was not shouting at me about the SSL connection. Everything was fine. But then came the surprise… I tried to log into Confluence (no I currently have no SSO 🙁 ). I let the browser enter my credentials and… got refused. I tried some more times manually with different combinations of users and passwords, checked my password store, checked CAPS LOCK,… but I did not get in. Since I disabled the „local admin“ of Confluence (due to being tight on the 10 user limit), I could also not check from inside Confluence.

What happened? After digging (this is what I often do 😉 ) through the settings, I stumbled over the Application Links section, which stated, that the connection could not be established due to SSL errors. Ah, OK, nice that Atlassian is recommending to install the JIRA SSL Add-on that helps with all that suff. Really all? For sure, not! Especially not with problems I encounter 🙁 . After digging deeper and deeper, I found, that the new chain of trust, 1&1 uses, is not setup in the JIRA keystore, the atlassian tools ship with their JRE. In contrast Chrome’s certificate store is updated tightly.

How to fix? First you need to find the Java keystore to add the CA certificate to be trusted. This is quite a bit difficult, if you do not know about the tool’s shipped JRE. So don’t hassle around with your distributions keystore, step into the folder where your tools are install (/opt/atlassian/ in my case) and do a

find . -type f -name cacerts

In my case, the list shows as follows:


I know, Bitbucket 5.2.2 is quite old (and the other tools, too), but keeping it up-to-date is quite hard for a spare-time setup. I believe, with an up-to-date installation, the problem would not have encountered. Let’s see, when I update… With this knowledge, it is quite easy (and some manual work) to add your cert to the keystore.

To get the certificate from your server, just do a

bash$ openssl s_client -showcerts -servername -connect </dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ${HOME}/

Then you can import that certificate into your Java keystore with the following command set (e.g. for confluence):

bash$ cd /opt/atlassian/jira/jre
bash$ bin/keytool -delete -alias \
-keystore lib/securety/cacerts -storepass changeit
bash$ bin/keytool -import -alias \
-keystore lib/securety/cacerts -storepass changeit \
-noprompt -file ${HOME}/

After doing that (I didn’t even need a restart of the tools), the application connections resurrect and you can log in to your confluence again.

The more convenient way is to write a little bash script that does the job. You can find mine here on Github. Feel free to improve it and issue some pull request if you think it’s worth to be shared.

Lessons learned

  1. Alsways keep a local admin account active in each and every Atlassian tool 😉
  2. Better use an automated SSL framework like Let’s encrypt. With it, you need to make the key rolling-update working from the beginning, not when it is too late (OK, this would not have helped me in my situation, nevertheless it is a good idea to do so)
  3. Document your problem solutions (which I do with scripts and this blog 🙂 )
  4. Don’t document the tool in the tool (e.g. this howto in Confluence), you will shoot both your feet 🙂
  5. Keep your tools up to date!

Windows (was) just a pain

Since I use Linux at home and love to develop embedded, backend and (web)-fronteds within a real operating system, I sometimes get crazy at work, when I just search for an alternative to a simple command line… So, what’s the alternative in Windows? The Command Prompt, then PowerShell or some specialized, magic, woodoo,… GUI application with the worst design ever seen in the universe and beyond?

OK, I see, you need an example 😉 Here it is one of my favorite: Syncronize a 100-GB-folder from one machine to another when one is at the end of the world, connected by avian carriers (see also RFC 2549 – IP over avian carriers) with a perceived rate of 2 bit per hour. 22 years ago, rsync was invented and serves every (unidirectional or pseude-bidirectional) syncronization desire with a sheer infinite amount of options… But, it is not available (directly) for windows…

Welp, some time ago, there was cygwin, which was driven by Red Hat. OK, it is still being developed, but somehow, I feel it is not serving my desires very well. At least not the desires a developer has. I also found MinGW and MSYS some years ago, but as I ran into trouble with wget and rsync when handling large files, I tested MSYS2 (it already includes MinGW-32 or MinGW-64, whichever you prefer). That was the starting point to test MSYS2 and to my surprise, it is exactly what makes my heart pound faster. All previous POSIX/Linux/Universe/Multiverse/… compatibility layers for Windows had some GUI to select packages, run updates,… But not MSYS2! It uses a pacman port. OK, I didn’t use pacman before when I was not forced to (I prefer APT), but it is COMMAND LINE and it runs on Windows (64-bit).

And even better, it did the ssh configuration well (a problem in old MSYS), so that you could generate and use an SSH key, which makes rsync even more powerful…

Hey guys of MSYS2, whenever you pass by in Germany (Erlangen), I will spend you some beer at Steinbach Bräu. This brewery is just to beer, what you are to Windows. Simply the greatest enrichtment 😉


So, if you would like to use rsync and wget in windows, just install MSYS2, do the obligatory update described on their page and execute the following:

pacman -S wget openssh rsync

Have fun with Bash, SSH, Rsync and all that other cool Linux tools on Windows!

Getting Started Embedded – Part II – The Embedded Project

Start embedded projects in an ordered way using the right tools

For many people, „project“ is a mysterious word and everybody understands something different about it. Sales department has a totally different understanding than engineers. Engineers see it quite similar to software developers, but there are facets their opinion differs. Just take a minute to think about what a project means to you.

So, after you got your own opinion, let’s first define, what a project means to me. At first it means nothing more than a vision, e.g. a customer’s product idea, to talk about. And with every vision, everybody has an individual understanding of it in the beginning.

The Information Collection Project

Welp, what is that? It has at first nothing to do with software, hardware or an embedded project. We start to collect information. Since we are living in the 21st century, we try to avoid using paper and put everything into „the cloud“. One way to organize such information is putting everything into a MS Word document… OK, this was a joke 🙂

I tend to use „the Atlassian tools“ (Jira, Confluence, Bitbucket) for that purpose. Not that I receive any revenue for that, but I did not find anything that can compete with it at the moment… Most important in my opinion at this stage is Confluence. But to start a „Space“ in Confluence, the more effective workflow is to start with a Jira project (again this word…).

If you create a Jira project, I suggest you use a template that contains user stories and bugs (e.g. Scrum or Kanban), and then add issue types (Tutorial) for requirements and impediments. Don’t try to optimize the workflow for every issue type, during the project is the perfect time to do it.

Now create a new Confluence space (Software Project Space template) and link it to your just created Jira project. This space is the container for all relevant project information. Neither is Jira only suited for software projects, nor is Confluence (with this software template). In fact, you can drive anything with it efficiently. I love it to steer my renovation projects around my house with it. (Yes, I’m a nerd 🙂 !) This is not classical project management with a tight schedule, it is simply collecting issues, linking them and pick them according to their priority and dependencies.

For those old stagers around us and for those new to agile principles or just interested people, I would recommend to have a look into Design Thinking. It gives you some ideas on how to collect the relevant information, define stakeholders,…

The Vision

One of the most important things, before we can talk about „the project“, we need to get a common understanding, what the inventor of that vision had in mind. If, and only if, the vision can be realized with hard- and software, we have an embedded project starting and many developers already have an idea, how it can be „solved“ as a collection of electronic parts and code. But this is the „solve a problem“ approach (engineers like that) to start a project, but very often not leading to customer’s satisfaction. We will start differently…

To get a common understanding, it is necessary to talk about the system’s requirements, making the vision more clear to all involved parties. At this point, it is the right time, to list possible stakeholders and the users of the potential product. Don’t forget to create meeting notes and other pages about your findings in Confluence and put every requirement into your Jira project. In Confluence you can create a nice page with a Jira-Table-Macro collecting requirements and also grouping them with filters for different stakeholders. You can use tags for that purpose or, even better, create Jira Service Desk accounts for your stakeholders (saving the costly developer accounts). For sure that requires the Jira Service Desk add-on (the smalles 3-Agent-Version should serve very well even for projects).

During this time of user and stakeholder analysis, the vision becomes more clear and a common understanding settles. Still it is not the time to carve the solution in stone, like many engineers/developers tend to (again, this is what they love). The vision will still evolve during „the project“ and also system requirements will do. Now it’s time for personas and use cases, to get an even better common understanding.

Use Cases

When we talk about use cases, we often only talk about users or customers of the end product. But you should never forget the other stakeholders. Even the development team can be a stakeholder, because they (in an agile environment) should be the responsible for quality and they for sure have specific requrements regarding QA (Quality Assurance, e.g. testing). BTW: Don’t forget a UART port in your hardware requirements if you go with headless embedded system. Otherwise the developers will kill you instantly when they move from eval board to prototype hardware.

If you have some uses cases collected, hopefully a few for every stakeholder, it’s time to move on. Don’t try to finalize the list, you will fail! It is much better to keep in mind, that the list will still evolve and steer your project to be agile. Try to prioritize them together with all stakehoders. Invite some avatars, if stakeholders can not be included (e.g. real end customers). Here personas come get handy.

The Project’s Disciplines

The list of requirement typed issues in Jira should be already extend across some pages and you identified the ones with most impact on your system design. Now the paths will split, depending on the resulting product. If it has some GUI, like many HMI projects tend to have nowadays, you should get some experienced Design Thinking engineer that possibly starts with paper prototypes or even more wired stuff. But this is not the focus of this post…

We know should define a few components. At first HW, SW and possible ME (Mechanical Engineering) will serve well. We can put more granularity in later on. Now try to assign your requirements to these components. If you did this, let the engineers of these disciplines have a look on every requirement that belongs to their component and let them comment on their understanding. Pull in the stakeholders (or their avatars) again to dicuss the requirements, should something be unclear.

Thougts about the Software Project

Now it is time to start the software project, isn’t it? Well, if you can tell, what a software project is! No, I don’t talk about the „Create project…“-button of your IDE. First we need to step back quite some distance to get a better overview on it.

We already defined some requirements above and assigned them components. The ones tagged with the SW-Component are the base for our user stories. Maybe we already defined some of them when we talked with the stakeholders, but now it’s time to check, if we have important requirements left, that have not been addresses by a story. But collecting these stories is another topic, we will not address in this post…

So, why I’m talking about that…? Because there should be some requirements and stories hanging around, that could conflict with the tools (and IDEs) we tend to use. If you are using e.g. Keil µVision and your have the requirement to do intensive testing (e.g. because there is some SIL-level required), then you should definitely rethink your decision using this IDE. Or if a customer requires a close feedback loop and high delivery rates (daily in extreme situations), then you need some continuous integration pipeline, which is hard to establich using GUI-driven IDEs.

Let me give an example. You have a project with many parties involved (maybe different companies in extreme). Think of a smart home system. Different companies design the sensor nodes, another the central (headless) unit, you design the HMI-unit and another company designs the GUI using a propritary tool for GUI design, sending you some pieces of API code and some static libraries to be linked in. You get a delivery every week and need to integrate it. The customer wants to test the new functionality of the GUI’s HMI (e.g. this week switch on light) as soon as possible. Many of the new functionalities and bug fixes from the GUI do not need your intervention, because they do not affect the API between embedded system and GUI. But not having an automated CI pipeline requires you to integrate the new GUI in your IDE and test it with every week’s delivery MANUALLY! I would hate your job from the bottom of my heart!

But you are an engineer and you love to solve problems immediately. Don’t make a project out of it, just raise an impediment (use this issue type in Jira) and find people to help solving it (Scrum-Master for communications and mental support, your boss for budget, IT employee for Jenkins,…).

The essence of this example is: If there are large parts of the development that could be automated using other tools (ot toolchains), start today changing your tools. Engineering should not consist of repetitive tasks, it should challenge your creativity with new tasks every week, every day,… every second. Switch over to CI and TDD, there is nothing giving you more satisfaction in your daily job as a software engineer. When mentioning TDD, already a few years old, but still very valueable is James W. Grenning’s book Test-Driven Development for Embedded C.

So before you start, choose the right toolchain. I highly recommend using GCC together with Eclipse and/or Makefiles. Why? Because it is platform independend, can be automated (even headless) and does not require a license. The licensing problem is not about the price, it is about the problem that you need one with anything or anybody involved in the development process. Even your Jenkins host needs a license, when you use commercial tools. And requiring a license makes it nearly impossible to spin up a fresh docker container for running a build. So, don’t shut the door on amazing technologies you don’t know yet but could make your life easier, like the wheel did six thousand years ago.

What’s next…?

In this post, I talked about starting an embedded project. Next post will be about starting the (non-GUI) embedded software of it.

Getting Started Embedded – Part I – The Toolchain


For getting started with any embedded development, the most important piece is the toolchain.

Many people suggest to use Keil, IAR, or some other fancy, professional, rocket-sience (and very expensive) IDE. In my opinion, that is just rubbish.

In former days, when tiny embedded controllers just have not been designed with compilers in mind, it was good to have some highly optimized compilers that could transfer a piece of C code to the ASM of these devices.

Nowadays, processors are designed with compilers in mind. Therefore, I would highly recommend to use GCC not only because of its low price tag (0 $), but because of its stunning community and active development. Since the community and many companies still putting so much effort into this piece of software, no single company can compete with an own closed source product.

Installing the ARM toolchain

There are many possible sources, where you can get GCC and all the tools you need to start. You can compile ARM-GCC yourself using your platform GCC, download the one from ARM directly, take a release from GNU MCU Eclipse and many many more…

Don’t know what to do? Just get kickstarted, and give XPM (a node/npm module) a try. Download and install node.js. The version does not matter too much. If you are not developing with node.js itself, better stick to the stable version.

When node.js is installed, install xpm and the ARM GCC toolchain (same for Windows, Linux & macOS):

me@diggerVM:~$ npm install xpm
me@diggerVM:~$ xpm install --global @gnu-mcu-eclipse/arm-none-eabi-gcc
me@diggerVM:~$ arm-none-eabi-gcc -v

OK, that was easy… If this worked, you maybe need some supporting tools. Best practice differs a bit, depending on your platform. For Linux, just install the build-essentials package (Debian dn ubuntu call it like this).

me@diggerVM:~$ sudo apt-get install build-essentials

For Windows (would also work for Linux, but I prefer the OS provided package), you can use xpm again:

C:\Users\me\>xpm install --global @gnu-mcu-eclipse/windows-build-tools

After this has finished, you possibly need to add the build tools to your PATH environment. You will find it in %APPDATA%\xPacks\@gnu-mcu-eclipse\windows-build-tools\2.11.1-1\.content\bin.

Testing your Toolchain

To test your setup, just clone, download,… the STM32 example project. Open a command line prompt, cd to the project directory and fire make:

me@diggerVM:~/GIT/stm32-example$ make

If this worked without errors, you have your toolchain up and running. Congratulations!

What’s next…

The next post will explain, how to setup STM32CubeMX and the Eclipse IDE to start developing own embedded applications effectively. Until now, there is not much difference to commercial IDEs and Toolchains from a workflow point of view. But don’t worry, we still have automation in mind and the goal is to have a CI-Pipeline running soon.

Cookie Banner von Real Cookie Banner