Skip to content

Latest commit

 

History

History
345 lines (212 loc) · 13.7 KB

developer-guide.md

File metadata and controls

345 lines (212 loc) · 13.7 KB
Raw

Developer guide

This guide helps you get started developing Grafana.

Dependencies

Make sure you have the following dependencies installed before setting up your developer environment:

macOS

We recommend using Homebrew for installing any missing dependencies:

brew install git
brew install go
brew install node@20
corepack enable

Windows

If you are running Grafana on Windows 10, we recommend installing the Windows Subsystem for Linux (WSL). For installation instructions, refer to our Grafana setup guide for Windows environment.

Download Grafana

We recommend using the Git command-line interface to download the source code for the Grafana project:

  1. Open a terminal and run git clone https://github.com/grafana/grafana.git. This command downloads Grafana to a new grafana directory in your current directory.
  2. Open the grafana directory in your favorite code editor.

For alternative ways of cloning the Grafana repository, please refer to GitHub's cloning a repository documentation.

Warning: Do not use go get to download Grafana. Recent versions of Go have added behavior which isn't compatible with the way the Grafana repository is structured.

Configure precommit hooks

We use pre-commit hooks (via lefthook) to lint, fix, and format code as you commit your changes. Previously the Grafana repository automatically installed these hook when you did yarn install, but they are now opt in for all contributors

Install the lefthook precommit hooks with:

make lefthook-install

To remove precommit hooks, run

make lefthook-uninstall

Note

Contributors working on the frontend are highly encouraged to install the precommit hooks, even if your IDE formats on save, so the .betterer.results file is kept up to sync.

Build Grafana

Grafana consists of two components; the frontend, and the backend.

Frontend

Before we can build the frontend assets, we need to install the dependencies:

yarn install --immutable

Troubleshooting: if you get the error The remote archive doesn't match the expected checksum for a dependency pulled from a link (e.g. "tether-drop": "https://github.com/torkelo/drop"): this is a temporary mismatch. To work around it (while someone corrects the issue), you can prefix your yarn install --immutable command with YARN_CHECKSUM_BEHAVIOR=update

After the command has finished, we can start building our source code:

yarn start

This command will generate sass theme files, build all external plugins, then build the frontend assets. Once yarn start has built the assets, it will continue to do so whenever any of the files change. This means you don't have to manually build the assets every time you change the code.

Troubleshooting: if your first build works, but after pulling updates you see unexpected errors in the "Type-checking in progress..." stage, these can be caused by the tsbuildinfo cache supporting incremental builds. You can rm tsconfig.tsbuildinfo and re-try.

Next, we'll build & run the web server that will serve the frontend assets we just built.

Backend

Build and run the backend by running make run in the root directory of the repository. This command compiles the Go source code and starts a web server.

Are you having problems with too many open files?

By default, you can access the web server at http://localhost:3000/.

Log in using the default credentials:

username password
admin admin

When you log in for the first time, Grafana asks you to change your password.

Building on Windows

The Grafana backend includes SQLite which requires GCC to compile. So in order to compile Grafana on Windows you need to install GCC. We recommend TDM-GCC. Eventually, if you use Scoop, you can install GCC through that.

You can build the back-end as follows:

  1. Follow the instructions to install the Wire tool.
  2. Generate code using Wire:
# Default Wire tool install path: $GOPATH/bin/wire.exe
<Wire tool install path> gen -tags oss ./pkg/server ./pkg/cmd/grafana-cli/runner
  1. Build the Grafana binaries:
go run build.go build

The Grafana binaries will be in bin\windows-amd64. Alternately, if you wish to use the make command, install Make for Windows and use it in a Unix shell (f.ex. Git Bash).

Test Grafana

The test suite consists of three types of tests: Frontend tests, backend tests, and end-to-end tests.

Run frontend tests

We use jest for our frontend tests. Run them using Yarn:

yarn test

Run backend tests

If you're developing for the backend, run the tests with the standard Go tool:

go test -v ./pkg/...

On Windows

Running the backend tests on Windows currently needs some tweaking, so use the build.go script:

go run build.go test

Run SQLLite, PostgreSQL and MySQL integration tests

By default, grafana runs SQLite, to run test with SQLite

go test -covermode=atomic -tags=integration ./pkg/...

To run PostgreSQL and MySQL integration tests locally, you need to start the docker blocks for MySQL and/or PostgreSQL test data sources by running make devenv sources=mysql_tests,postgres_tests. When your test data sources are running, you can execute integration tests by running:

make test-go-integration-mysql

and/or

make test-go-integration-postgres

Run end-to-end tests

Grafana uses Cypress to end-to-end test core features. Core plugins use Playwright to run automated end-to-end tests. You can find more information on how to add end-to-end tests to your core plugin here

Running Cypress tests

To run all tests in a headless Chromium browser.

yarn e2e

By default, the end-to-end tests start a Grafana instance listening on localhost:3001. To use a different URL, set the BASE_URL environment variable:

BASE_URL=http://localhost:3333 yarn e2e

To follow all tests in the browser while they're running, use yarn e2e:debug

yarn e2e:debug

To choose a single test to follow in the browser as it runs, use yarn e2e:dev

yarn e2e:dev

To run the Playwright tests:

Note: If you're using VS Code as your development editor, it's recommended to install the Playwright test extension. It allows you to run, debug and generate Playwright tests from within the editor. For more information about the extension and how to install it, refer to the Playwright documentation.

Each version of Playwright needs specific versions of browser binaries to operate. You will need to use the Playwright CLI to install these browsers.

yarn playwright install chromium

To run all tests in a headless Chromium browser and display results in the terminal.

yarn e2e:playwright

For a better developer experience, open the Playwright UI where you can easily walk through each step of the test and visually see what was happening before, during and after each step.

yarn e2e:playwright:ui

To open the HTML reporter for the last test run session.

yarn e2e:playwright:report

Configure Grafana for development

The default configuration, defaults.ini, is located in the conf directory.

To override the default configuration, create a custom.ini file in the conf directory. You only need to add the options you wish to override.

Enable the development mode, by adding the following line in your custom.ini:

app_mode = development

Add data sources

By now, you should be able to build and test a change you've made to the Grafana source code. In most cases, you'll need to add at least one data source to verify the change.

To set up data sources for your development environment, go to the devenv directory in the Grafana repository:

cd devenv

Run the setup.sh script to set up a set of data sources and dashboards in your local Grafana instance. The script creates a set of data sources called gdev-<type>, and a set of dashboards located in a folder called gdev dashboards.

Some of the data sources require databases to run in the background.

Installing and configuring databases can be a tricky business. Grafana uses Docker to make the task of setting up databases a little easier. Make sure you install Docker before proceeding to the next step.

In the root directory of your Grafana repository, run the following command:

make devenv sources=influxdb,loki

The script generates a Docker Compose file with the databases you specify as sources, and runs them in the background.

See the repository for all the available data sources. Note that some data sources have specific Docker images for macOS, e.g. nginx_proxy_mac.

Build a Docker image

To build a Docker image, run:

make build-docker-full

The resulting image will be tagged as grafana/grafana:dev.

Note: If you are using Docker for macOS, be sure to set the memory limit to be larger than 2 GiB. Otherwise, grunt build may fail. The memory limit settings are available under Docker Desktop -> Preferences -> Advanced.

Troubleshooting

Are you having issues with setting up your environment? Here are some tips that might help.

IDE configuration

Configure your IDE to use the Typescript version from the Grafana repository. The version should match the Typescript version in the package.json file, and is typically at the path node_modules/.bin/tsc.

Previously Grafana used Yarn PnP to install frontend dependencies, which required additional special IDE configuration. This is no longer the case. If you have custom paths in your IDE for ESLint, Prettier, or Typescript, you can now remove them and use the defaults from node_modules.

Too many open files when running make run

Depending on your environment, you may have to increase the maximum number of open files allowed. For the rest of this section, we will assume you are on a Unix like OS (e.g. Linux/macOS), where you can control the maximum number of open files through the ulimit shell command.

To see how many open files are allowed, run:

ulimit -a

To change the number of open files allowed, run:

ulimit -S -n 4096

The number of files needed may be different on your environment. To determine the number of open files needed by make run, run:

find ./conf ./pkg ./public/views | wc -l

Another alternative is to limit the files being watched. The directories that are watched for changes are listed in the .bra.toml file in the root directory.

To retain your ulimit configuration, i.e. so it will be remembered for future sessions, you need to commit it to your command line shell initialization file. Which file this will be depends on the shell you are using, here are some examples:

  • zsh -> ~/.zshrc
  • bash -> ~/.bashrc

Commit your ulimit configuration to your shell initialization file as follows ($LIMIT being your chosen limit and $INIT_FILE being the initialization file for your shell):

echo ulimit -S -n $LIMIT >> $INIT_FILE

Your command shell should read the initialization file in question every time it gets started, and apply your ulimit command.

For some people, typically using the bash shell, ulimit fails with an error similar to the following:

ulimit: open files: cannot modify limit: Operation not permitted

If that happens to you, chances are you've already set a lower limit and your shell won't let you set a higher one. Try looking in your shell initialization files (~/.bashrc typically), if there's already an ulimit command that you can tweak.

Getting AggregateError when building frontend tests

If you encounter an AggregateError when building new tests, this is probably due to a call to our client backend service not being mocked. Our backend service anticipates multiple responses being returned and was built to return errors as an array. A test encountering errors from the service will group those errors as an AggregateError without breaking down the individual errors within. backend_srv.processRequestError is called once per error and is a great place to return information on what the individual errors might contain.

Next steps