migrate-sdk-typespec

name	migrate-sdk-typespec
description	**WORKFLOW SKILL** - Update SDK project and TypeSpec project, regenerate SDK, to fix error on SDK project. USE FOR: "migrate sdk lib for <tsp-project>".

Skill Instructions

Request

The request would come in the form of:

• "Migrate sdk lib for <tsp-project>"

The <tsp-project> would be a relative path to a folder in specs repo. A typical form of it can be "specification/web/resource-manager/Microsoft.Web/AppService".

Checklist

• DO NOT use force push. Always create a new commit for each change, to make sure the history is clear and traceable.
• Files in "autorest.java" repo and folder are not relevant to this skill. Do not read or modify any files in "autorest.java" repo and folder, except this skill folder.
• For TypeSpec project, only update "client.tsp" file and "tspconfig.yaml" file.
• When updating "client.tsp", always use "java" scope, and add new lines to the end of the file.
• For SDK project, do not modify Java file that contains // Code generated in the header comment. These are Java files generated by the emitter from TypeSpec project, and should not be modified by agent individually.
• Run tsp format <tsp_file> command, whenever you modify the .tsp file, to make sure the format is correct.

Required repositories

• Azure/azure-sdk-for-java, cloned at "../azure-sdk-for-java" Call this folder "sdk repo" for short.
• Azure/azure-rest-api-specs, cloned at "../azure-rest-api-specs" Call this folder "specs repo" for short.

You have full access to these locally cloned repositories/folders.

Steps

Before you start

Make sure both sdk repo and specs repo are on a migrate-<tsp-project> branch. If not, checkout latest "main", and make a new branch from it.

Run npm ci in "eng/common/tsp-client" folder in sdk repo to make sure the tsp-client is installed.

Run npm ci in specs repo to make sure all dependencies are installed.

Find the TypeSpec project in specs repo

Find a "tspconfig.yaml" file under <tsp-project> folder in specs repo. This file indicates the folder is a TypeSpec project.

<tsp-path> refers to the full path of this TypeSpec project folder in specs repo.

Correct "tspconfig.yaml"

Only modify the YAML under "options"/"@azure-tools/typespec-java" property.

• Remove ".generated" segment from "emitter-output-dir" and "namespace" properties.
• Make sure these options are set

    premium: true
    generate-tests: false
    client-side-validations: true
    enable-sync-stack: false
  

• If unknown appears in .tsp files, add option use-object-for-unknown: true
• If uuid appears in .tsp files, add option uuid-as-string: false
• Read "readme.java.md" if available, add options of rename-model, add-inner, remove-inner, preserve-model, resource-collection-associations into "tspconfig.yaml". See options.ts for details of these options.

Find the project in sdk repo

Now you have the correct "tspconfig.yaml" file, you can find the corresponding SDK project in sdk repo.

Read "emitter-output-dir" property and "service-dir" property (use "options"/"@azure-tools/typespec-java"/"service-dir" if available, otherwise use "parameters"/"service-dir"), combine them for <sdk-project>. The typical form of it can be "sdk/appservice/azure-resourcemanager-appservice".

<sdk-path> refers to the full path of the <sdk-project> folder.

<sdk-service> refers to the middle segment of the "emitter-output-dir" property in "tspconfig.yaml" file. The typical form of it can be "appservice".

<sdk-package> refers to the last segment of the "emitter-output-dir" property in "tspconfig.yaml" file. The typical form of it can be "azure.resourcemanager.appservice".

Initial mitigation

Here is a list of commonly encountered names used in .tsp that need to be mitigated via @@clientName(<model-or-property-or-method>, "<correct-name>", "java");:

eTag: etag
userName: username
metaData: metadata
timeStamp: timestamp
hostName: hostname
webHook: webhook
coolDown: cooldown
ID: id
IP: ip
ARM: arm
API: api

Use PascalCase for model names. E.g. ResponseMetaData to ResponseMetadata. Use camelCase for property and method names. E.g. acrUserManagedIdentityID to acrUserManagedIdentityId.

Iterate to fix compile errors

Generate the Java code

Run

python eng/automation/generate.py -s <sdk-service> -c <tsp-path>/tspconfig.yaml -v <sdk-version>

in SDK repo root folder.

This script will generate the Java code, and build the Java lib. It may encounter compile error that you need to fix in next step.

PS: this script can take a while to run. If you wait for the command, wait a maximum of 20 minutes.

Fix compile error

See Solve Compile Error for common compile errors and mitigation.

If you work on it for a while, but does not make progress, pause and summarize the errors.

Modify "module-info.java"

Add this line to "module-info.java" file.

opens com.azure.resourcemanager.<sdk-service>.implementation.models to com.azure.core;

Iterate to fix breaking changes

Build the Java lib

Run

mvn install -pl com.azure.resourcemanager:<sdk-package> -am -DskipTests

in SDK repo root folder.

It may encounter error from revapi check. This means there are breaking changes compared with the previous version, and you need to either suppress, or fix in next step.

Suppress or fix revapi error

See Solve Revapi Error for mitigation.

The goal here is not to suppress all errors so that build can pass. It is to suppress the few breaks that is allowed (e.g. move Paged model to implementation package, remove ctor and setter for read-only model, change of base class). In the meantime, figure out some breaks that can easily be mitigated by a rename in "client.tsp", and fix those breaks instead of suppressing them.

It is known that there will be some hard to mitigate breaks, e.g. enum changed to ExpandableStringEnum, that need decision from user on how to proceed. Do report these breaks that does not have clear mitigation, and wait for user decision before proceeding.

When you finished this step, pause and output report on remaining errors. Use the git diff with "main" branch, to provide the details on what you think be the cause of the error.

Compare client APIs

Write down all the *Client in fluent package, and their public APIs.

Compare it with the *Client and APIs in main branch. List the differences.

Update "CHANGELOG.md"

This step should be invoked by user, not by agent.

Remove all generated content in the section of latest lib version of "CHANGELOG.md" file. Also, update the date to "Unreleased".

Then, add a new "Breaking Changes" sub section in this lib version section. The general items of this would be supplied by user.

### Breaking Changes

- Removed `<class>` class. <reason>
- Removed `<method>` method from `<class>` class. <reason>
- Changed `<method>` method to `<new_method>` in class `<class>` class. <reason>
etc.

Create pull request and finalize the sdk lib

This step should be invoked by user, not by agent.

Verify filename is consistent on GitHub and on filesystem

Make sure the filename on GitHub is consistent with the filename on filesystem (typical reason is that Windows filename is case-insensitive, but git is case-sensitive). If not, use git mv command to rename the file on git to make them consistent.

Create pull request on specs repo

Create a draft pull request in specs repo.

Add label "PublishToCustomers", "ARMSignedOff", "BreakingChange-Go-Sdk-Approved", "BreakingChange-JavaScript-Sdk-Approved", "BreakingChange-Python-Sdk-Approved" to the draft PR.

Update "tsp-location.yaml" file, create pull request on sdk repo

When the pull request on specs repo is ready, get the SHA of the last commit.

Update the "commit" property of the "tsp-location.yaml" file in sdk repo, to make it point to this SHA.

Commit the change, and create a draft pull request in sdk repo.

Update "tsp-location.yaml" file the last time

When the pull request on specs repo is merged, update the "commit" property of the "tsp-location.yaml" file in sdk repo to point to the SHA on latest "main" branch of specs repo.