# Simorgh
[<img src="https://devin.ai/assets/askdeepwiki.png" alt="Ask DeepWiki about Simorgh" height="20"/>](https://deepwiki.com/bbc/simorgh)
[](https://snyk.io/test/github/bbc/simorgh)
[](https://bbc.github.io/simorgh/)
BBC World Service News websites are rendered using Simorgh, a ReactJS based application. Simorgh also renders AMP news article pages for World Service and Public Service News.
Simorgh provides a fast and accessible web experience used by millions of people around the world each month ([see list of websites using Simorgh](https://github.com/bbc/simorgh/blob/latest/docs/Simorgh-Release-Info.mdx)). It is regularly maintained and well documented, and we welcome open source contributors.
Simorgh is primarily maintained by the BBC News Web Engineering teams. It delivers highly trusted news to readers all over the world, currently in ([41 languages](https://www.bbc.com/ws/languages)). We support a wide range of devices and care deeply about scale, performance, and accessibility. We work in agile, flexible teams, and have an exciting roadmap for future development.
## Documentation index
Please familiarise yourself with our:
- [Code of conduct](https://github.com/bbc/simorgh/blob/latest/.github/CODE_OF_CONDUCT.md)
- [Coding Standards](https://github.com/bbc/simorgh/blob/latest/docs/Coding-Standards/README.md)
- [Contributing guidelines](https://github.com/bbc/simorgh/blob/latest/CONTRIBUTING.md)
- [Guide to Code Reviews](https://github.com/bbc/simorgh/blob/latest/docs/Code-Reviews.mdx)
- [GPG Signing Guide](https://github.com/bbc/simorgh/blob/latest/docs/GPG-Signing-Guide.mdx)
- [Primary README](https://github.com/bbc/simorgh/blob/latest/README.md) (you are here)
- [Recommended Tools](https://github.com/bbc/simorgh/blob/latest/docs/Recommended-Tools.mdx)
- [Troubleshooting](https://github.com/bbc/simorgh/blob/latest/docs/Troubleshooting.mdx)
NB there is further documentation colocated with relevant code. The above list is an index of the top-level documentation of our repo.
## Before Installation
Please read:
[CONTRIBUTING.md](https://github.com/bbc/simorgh/blob/latest/CONTRIBUTING.md)
## Installation
Install Node. [https://nodejs.org/en/](https://nodejs.org/en/). We use the version specified in `.nvmrc` and if you have a node version manager (nvm) you can run the following script to automatically change to the project supported version.
```
nvm use
```
### Install Yarn
The Simorgh project uses Yarn for package management. It is recommended to install Yarn through the npm package manager, which comes bundled with Node.js when you install it on your system. To install Yarn, run this command:
```
npm install --global yarn
```
Then you can run the following commands to install Simorgh
```
git clone git@github.com:bbc/simorgh.git
cd simorgh
yarn install
```
## Local Development
Export a relevant BFF_PATH so that simorgh can connect to the relevant fabl instance:
```
# For a local instance of fabl
export BFF_PATH="http://localhost:3210/module/simorgh-bff"
```
Start the application:
```
cd ws-nextjs-app
yarn dev
```
The nextJS application will start on [http://localhost:7081](http://localhost:7081).
Once the application is running, you can visit a valid route, e.g. http://localhost:7081/pidgin.
### Secret Environment Variables
Some local-development commands used by BBC staff require additional environment variables that are not stored in this repository. BBC staff can follow this [guide](https://github.com/bbc/simorgh-infrastructure/tree/latest/scripts/addShellProfileEnvVariables) (requires access) to set them on your machine.
### Feature toggles
Some features are hidden/shown based on feature toggles set remotely via BBC internal systems. By default when running the application locally however, these values are not fetched remotely and are set in [these config files](https://github.com/bbc/simorgh/tree/latest/src/app/lib/config/toggles). As such, these may differ between environments. Please read the [toggles documentation](https://github.com/bbc/simorgh/blob/latest/src/app/lib/config/toggles/README.md) for more information and for instructions on how to fetch remote toggle values locally.
### Article pages
Article pages are served at routes of the format `/news/articles/:id` where id is the asset ID generated by the Content Management System.
FYI: [Article explaining the BBC's use of ids in URL](https://www.smashingmagazine.com/2014/05/responsive-design-begins-with-the-url/)
These two News articles are available on the Test environment of our CMS, as well as locally, so are often used for testing:
- [http://localhost:7081/news/articles/c6v11qzyv8po](http://localhost:7081/news/articles/c6v11qzyv8po)
- [http://localhost:7081/persian/articles/c4vlle3q337o](http://localhost:7081/persian/articles/c4vlle3q337o).
We are also serving AMP HTML pages at the route `/news/articles/:id.amp` [https://www.ampproject.org](https://www.ampproject.org)
- [http://localhost:7081/news/articles/c6v11qzyv8po.amp](http://localhost:7081/news/articles/c6v11qzyv8po.amp)
- [http://localhost:7081/persian/articles/c4vlle3q337o.amp](http://localhost:7081/persian/articles/c4vlle3q337o.amp).
Services with variants can't be accessed using the format above, instead the variant must be provided in the URL.
- [http://localhost:7081/zhongwen/articles/c3xd4x9prgyo/simp](http://localhost:7081/zhongwen/articles/c3xd4x9prgyo/simp)
- [http://localhost:7081/zhongwen/articles/c3xd4x9prgyo/simp.amp](http://localhost:7081/zhongwen/articles/c3xd4x9prgyo/simp.amp).
### Home pages
World Service home pages are served in the format `/:service` where `service` represents a World Service site:
- [http://localhost:7081/igbo](http://localhost:7081/igbo)
- [http://localhost:7081/pidgin](http://localhost:7081/pidgin)
Services with variants can't be accessed using the format above, instead the variant must be provided in the URL.
- [http://localhost:7081/zhongwen/simp](http://localhost:7081/zhongwen/simp)
- [http://localhost:7081/zhongwen/simp.amp](http://localhost:7081/zhongwen/simp.amp).
World Service home pages do not support AMP.
### Topic Pages
Topic pages use internal BBC APIs that are not publicly accessible. This can cause the following warnings to appear when developing locally:
```
No BFF_PATH set as environment variable, you will not have access to topics
```
Internal developers who need to work on topic pages locally should contact the team for access. See [Secret Environment Variables](#secret-environment-variables) for how to set the required environment variables.
### Other page types
You can find other pages types by looking through our routes and their associates regexes, but we suggest you start with the above then have a look at the core of the application to understand and find the other routes.
[See a list of local route examples here](https://github.com/bbc/simorgh/blob/latest/docs/Simorgh-Release-Info.mdx).
### Storybook (UI Development Environment/Style Guide)
We use Storybook for developing components in isolation from the Simorgh Application. You can access this at [https://bbc.github.io/simorgh/](https://bbc.github.io/simorgh/)
To run locally `yarn storybook`, it will then be available at [http://localhost:9001/](http://localhost:9001/). Introduction to and documentation for Storybook is here: [https://storybook.js.org/basics/introduction/](https://storybook.js.org/basics/introduction/).
When viewing Video stories locally, make sure to use a BBC domain, as outlined in the [changing request location section](https://github.com/bbc/simorgh#changing-request-location). Video will not work in the hosted version of Storybook linked above for this reason.
We also use [Chromatic QA](https://docs.chromaticqa.com/) to run cross-browser testing on our stories.
Storybook compiles JavaScript, TypeScript and JSX with SWC, configured in [`.storybook/main.ts`](./.storybook/main.ts). It uses Emotion's JSX runtime and SWC plugin.
When updating Storybook's SWC compiler or `@swc/plugin-emotion`, check their [plugin compatibility](https://swc.rs/docs/plugin/selecting-swc-core) and validate Emotion-styled stories as well as SCSS Modules and MDX documentation.
Please also note that if you would like to see the components rendered with our fonts, you will need to force a repaint of the canvas. This is because our fonts all have the `font-display` property of `optional` or `swap` in accordance with the respective loading strategies here: https://ws-downloads.files.bbci.co.uk/fonts/index.html. The easiest way to force a repaint is just to move the divider between the preview window the and `Knobs` section or resize the browser window.
### Configuring the application to run on a local network
If you want to host the application to be accessible through your local network, follow the instructions [here](https://github.com/bbc/simorgh/blob/latest/docs/A11y-Testing-Cross-Device.mdx).
## Production build locally
To run this application locally with a production build, run:
`yarn build && yarn start`.
We use `yarn build` locally which bundles the application pointing at localhost for data and static assets.
## Using environment builds locally
This is mainly used for debugging `latest` using the TEST and LIVE environment bundles. Ensure that the bundles exist in the static asset location for the correct environment before starting to debug.
To run TEST bundles on localhost:
- In `envConfig/test.env` change the values of:
- `LOG_DIR='/var/log/simorgh'` to `LOG_DIR='log'`
- Then run `rm -rf build && yarn build:test && yarn start`
- Visit a test article: http://localhost:7081/news/articles/c0g992jmmkko
To run LIVE bundles on localhost:
- In `envConfig/live.env` change the values of:
- `LOG_DIR='/var/log/simorgh'` to `LOG_DIR='log'`
- Then run `rm -rf build && yarn build:live && yarn start`
- Visit a live article: http://localhost:7081/news/articles/c8xxl4l3dzeo
## Changing request location
Some features perform differently dependant on whether a user is located within the UK or internationally. You can explicitly request a specific version by accessing Simorgh via a specific localhost BBC domain:
- UK version: [http://localhost.bbc.co.uk:7081/news/articles/c0000000001o](http://localhost.bbc.co.uk:7081/news/articles/c0000000001o)
- International version: [http://localhost.bbc.com:7081/news/articles/c0000000001o](http://localhost.bbc.com:7081/news/articles/c0000000001o)
If these urls do not work, you may need to add a hosts file entry (`/etc/hosts` or `C:\Windows\System32\drivers\etc\hosts`):
```
127.0.0.1 localhost.bbc.co.uk
127.0.0.1 localhost.bbc.com
```
## Production build on CI
On deployment `make buildCi` is run in the CI environment which creates bundles for both the `test` and `live` environments. On the two environments the `.env.test` or `.env.live` files overwrite the `.env` file which is used to run the application with the correct bundles.
### Bundle analysis reports
Every run of `yarn build` will update the bundle analysis files in the repo. To view a breakdown of the bundle size, open the generated html report in a browser `./reports/webpackBundleReport.html` This is generated via `webpack-bundle-analyzer`. The data is also available as json `./reports/webpackBundleReport.json`.
## Tests
### Linting and unit tests
We have linting with the [Airbnb styleguide](https://github.com/airbnb/javascript/tree/master/react) and we use [Prettier](https://github.com/prettier/prettier) as a code formatter. They can be run with `yarn test:lint`.
ESLint parses JavaScript and JSX with its built-in parser and TypeScript and TSX with `@typescript-eslint/parser`.
Both `yarn test:lint` and `yarn test:lint:fix` use a content-based cache in `.eslintcache` to skip unchanged files on subsequent runs. The existing lint scope and rules are unchanged. Use `yarn test:lint --no-cache` for a full uncached run, for example after updating dependencies or Prettier configuration. ESLint's per-file cache does not track changes to imported files, so use an uncached run when checking cross-file changes.
We have [Jest](https://facebook.github.io/jest) unit tests that can be ran with `yarn test:unit` for the root projects and `yarn test` from the `ws-nextjs-app` directory for the Next.js project.
The repository has one root `jest.config.ts` containing seven named projects: `Unit Tests`, `Unit Tests (Client)`, `Puppeteer Tests`, `Unit Tests (Next)`, `Integration Tests - Canonical`, `Integration Tests - AMP` and `Integration Tests - Lite`. The root and Next project groups use the same SWC compiler with separate project-group options and test environments. Root projects use SWC without app-derived compiler options; Next projects retain the app-derived settings from the Next workspace.
Root and Next unit and integration projects share [src/testHelpers/setupTests.js](./src/testHelpers/setupTests.js) for DOM matchers, Web APIs, crypto mocks, `setImmediate` and console handling. All these projects fail on unexpected console errors, with the existing warning suppressions retained. Browser-specific error filtering only runs when a window is available, so Node-only API tests use the same file. Location mocking is left to individual tests rather than replacing the environment's location globally. Root-only mocks and fixture environment values still run earlier through [src/testHelpers/jest-setup.js](./src/testHelpers/jest-setup.js); Next integration tests retain their real fetch. Puppeteer keeps its separate setup.
`yarn test:unit` selects the two root unit projects. The Next workspace's `yarn test` selects only `Unit Tests (Next)`, and its integration scripts select their named platform projects. All workspace commands explicitly use the root configuration so running them from either repository directory resolves the same projects. The root integration command continues to manage the app lifecycle through `ws-nextjs-app/integration/utils/runTests.ts`.
Resolving the combined configuration initializes the Next app configuration and its `.env` files even when only root projects are selected. This is an intentional trade-off of keeping one configuration, so root test commands may observe app environment values during configuration resolution. A single global `jest-junit` reporter writes results to the repository root's `reports/jest` directory, regardless of the invocation directory.
SWC preserves read-only module exports. Tests that need to spy on or replace imported exports use explicit `jest.mock` factories to provide mockable exports. Existing implementations can be preserved with `jest.requireActual` and object spreading.
### End-to-end tests
#### Main application
We use [Cypress](https://www.cypress.io/) for our end-to-end tests. To run the [smoke tests](https://github.com/bbc/simorgh/tree/latest/cypress/integration#how-our-cypress-tests-work) locally, run this single command:
```
yarn test:e2e
```
It will spin up a production server on port 7081 and run the Cypress tests against that.
To run the smoke tests interactively, run:
```
yarn test:e2e:interactive
```
This loads a user interface which easily allows for individual tests to be run alongside a visual stream of the browser, as the tests run.
#### Environment variables
There are several environment variables you can use with our test suite, which are:
| Environment variable | Effect | Possible values |
| -------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- |
| CYPRESS_ONLY_SERVICE | Restricts to running only the specified service | A single service i.e. `CYPRESS_ONLY_SERVICE=urdu` |
| CYPRESS_APP_ENV | Runs the tests in a specific environment | `test`, `local`, `live` |
| CYPRESS_SMOKE | Runs only smoke tests if true | `true`, `false` |
| CYPRESS_UK | See [running e2es in the UK against Live](https://github.com/bbc/simorgh/#running-e2e-in-the-uk-against-live) | `true`, `false` |
| CYPRESS_SKIP_EU | See [running e2es outside EU](https://github.com/bbc/simorgh/#running-e2e-outside-eu) | `true`, `false` |
These commands can be run in combination.
#### Full suite of tests
The default way to run the e2e suite aka `yarn test:e2e` or `yarn test:e2e:interactive` runs a subset of our tests, otherwise know as _smoke tests_. To run the full suite:
`CYPRESS_SMOKE=false yarn test:e2e`
#### Limiting scope of runs
Tests can be restricted to only run for a single service by specifying it using the `CYPRESS_ONLY_SERVICE` environment variable. For example:
```
CYPRESS_ONLY_SERVICE=urdu yarn test:e2e
```
To run only a particular spec it is necessary to invoke Cypress directly. First ensure Simorgh is already running in another tab and then run (for example, to only run article tests):
```
npx cypress run --spec cypress/integration/pages/articles/index.js
```
Further details on using the Cypress CLI can be found at https://docs.cypress.io/guides/guides/command-line.html
#### Running e2e in the UK against LIVE
**This affects developers based in the UK only (but may affect you if you're using a VPN routing through the UK)**
Cypress .visit() function is locked to visiting a single domain per test. This becomes problematic when you launch the e2e tests from within the UK, due to redirects from `.com` to `.co.uk`. By default cypress tests will run as if they were ran outside of the uk. In order to run these tests from the UK you have to pass in the `UK` Cypress environment variable to the tests. This will replace the URL endings to `.co.uk`, which will allow you to run these tests successfully.
Here is an example command:
```
CYPRESS_APP_ENV=test CYPRESS_UK=true CYPRESS_SMOKE=true yarn cypress
```
#### Running e2e outside EU
**This affects developers based out of the EU (but may affect you if you're using a VPN routing through a country not in the EU)**
Running Cypress tests outside the EU will not show the EU consent banners on AMP, and this may cause some tests to fail. Set `CYPRESS_SKIP_EU=true` to prevent these tests from running when outside the EU.
An example command will be:
```
CYPRESS_SKIP_EU=true yarn cypress:interactive
```
The following command runs both simorgh and cypress:
```
CYPRESS_APP_ENV=local CYPRESS_UK=true CYPRESS_SMOKE=true yarn test:e2e
```
CYPRESS_APP_ENV can also be set equal to 'test' and 'live'.
CYPRESS_SMOKE can be true or false. It is true by default and runs a specific subset of tests.
### Lighthouse Best Practice tests
We use [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci) to test our pages. A single config, [`lighthouse/lighthouserc.js`](./lighthouse/lighthouserc.js), is shared by local runs, GitHub Actions and our CodeBuild pipelines. `LIGHTHOUSE_APP_ENV` selects which URLs are audited and which thresholds apply.
To run it against a local Next.js server on port 7081:
```
yarn lighthouse
```
To run it against the test or live environments (as CodeBuild does):
```
LIGHTHOUSE_APP_ENV=test yarn lighthouse
```
Local reports are written to `.lighthouseci/`.
### Why is it called Simorgh?!
Named Simorgh after the Persian mythological bird. The Simorgh is the amalgam of many birds (and in some accounts other animals) into one.
Happily, a metaphor which seemed apt for offering all BBC articles in one solution is perhaps now even more appropriate as the application evolves to support more content types. It’s also a clear reference to the international nature of our teams, but also to the desire to ensure articles (and everything which has followed) works for users in all languages the BBC supports.
It is also a unique name which is practical and, more superficially, the bird is very pretty.