Company

Recap Functional - Open Source

Christoph Thale|
#company#recap

What on earth is Open Source?

Open source in software engineering is a collaborative approach where the source code of a program is freely available for anyone to view, modify, and distribute. It fosters transparency, community-driven innovation, and peer review, leading to more reliable and secure software. Open Source projects range from operating systems to web browsers and benefit businesses by reducing costs and promoting interoperability. Overall, open source embodies a culture of collaboration and democratization in software development.

Some of the most famous open source software projects are the Linux Operating System, the Mozilla Firefox Web Browser, LibreOffice Suite, Git Version Control System, Kubernetes and Docker.

Why we do it

Contributing to open source software (oss) is not just about coding. It is about giving back, personal growth, enjoyment, and community building. Let me clarify this:

  • Personal growth and enjoyment: Contributing to oss is fun! You get to work on projects you love or use, adding features or fixing bugs that directly impact your experience. It’s satisfying to see your contributions make a difference.

  • Learning opportunities: Oss projects offer endless learning opportunities. Whether you’re experimenting with new programming techniques, diving deeper into libraries, or collaborating with others, you’re constantly expanding your skills and knowledge.

  • Giving back to the community: Open-source software is built on the contributions of countless individuals. By giving back, you help sustain and improve projects that others rely on. It’s a way of saying thanks for the free tools and resources you’ve benefited from.

  • Networking and collaboration: Contributing to open source projects connects you with like-minded individuals worldwide. You build relationships, share ideas, and learn from each other’s experiences, enriching both your personal and professional networks.

Where we contribute

Yannick

Among other things, for our customer, we are running a GraphQL Mesh in order to integrate the many different endpoints we are providing. From my experience, I know how annoying it is to search in documentation and therefore, I decided to update some of it, which includes fixing typos and adding new stuff.

Another great GraphQL tool is GraphQL Hive, which we are using to track user’s requests and generate insights about error frequency or whether we can do schema updates without problems. In order to use GraphQL Hive, we have also wanted to expose Prometheus metrics. However, from the documentation it was not obvious on which port prometheus was running and it was also not possible to change this. As in some of our environments, we do need different configurations for this, we have added these options, such that you may now change the Prometheus port and the documentation states what the default ports are, saving others a lot of time.

Similarly, embedding Hive into GraphQL Mesh should be easy to do. However, you may only set the GraphQL Hive’s URL in the configuration when building the mesh. Normally, this should just be a configuration change, as between the deployed versions and local development. Therefore, we have added this Option, to set the URL to GraphQL Hive via environment variables.

Michael

As Yannick mentioned, we use GraphQL Mesh for our customer. We have around 50 entities and over 100 connections in the graph, which are auto-generated schemas from XSD files. For the generation, we use Prisma, the prisma-nestjs-graphql generator, and some black magic. Each property usually has a comment explaining the business reason behind it. We noticed that some comments, especially for nested types and enums, were missing. For us, this was quite an issue because the consumers of the API needed that information to understand the complex graph model. We have contributed to the Prisma Engine Rust codebase to provide these comments to their generators.

Sometimes, contribution isn’t all about contributing code or reviewing it; sometimes it’s just about pushing the right people to do something. That’s the case for the Nestjs Azure Database library where we pushed the contributor to prioritize upgrading to the newest Azure SDK version. Not a small change to make, and highly requested by others, this issue was open for quite some time. Writing the maintainer an email, describing our need, asking very nicely, and being persistent might have done the trick to make the issue a top priority. Good for us, the library, and the other 20 people who asked.

Christoph

In a project with our customer where we have deployed a large Azure Data Explorer, we noticed that setting up, managing, or deleting resources like tables, update policies, materialized views, and so on was quite cumbersome. Naturally, everyone is now thinking about Infrastructure as Code, just like us, but we couldn’t find the desired functionalities in the official Terraform provider (azurerm). Fortunately, we stumbled upon the open-source ADX Terraform provider by favoretti. This was exactly the approach we were looking for at that time. Wonderful, but unfortunately, some small pieces of functionality were missing here and there.

Therefore, we went ahead and added those missing features ourselves. True to the motto “If we find it useful, surely others will too”. Specifically, over the time I implemented the following:

  • added the option to add two more parameters (concurrency and maxSourceRecordsForSingleIngest) to the Materialized Views creation
  • added terraform import functionality to adx_function
  • added the possibility to update the property autoUpdateSchema for given Materialized Views
  • integrated new resource: Column Encoding Policy
  • integrated new resource: Continuous Export
  • integrated new resource: External Table
  • enabled to create an Update Policy with a Managed Identity
  • updated docs

That’s all for now. Thank you and I will see you in the next blog article.

Back to Blog