功能介紹:

錄音並即時獲取RAW的音頻格式數據,利用WebSocket上傳數據到服務器,並即時獲取語音識別結果,參考文檔使用AudioCapturer開發音頻錄製功能(ArkTS),更詳細接口信息請查看接口文檔:AudioCapturer8+@ohos.net.webSocket (WebSocket連接)

知識點:

  1. 熟悉使用AudioCapturer錄音並即時獲取RAW格式數據。
  2. 熟悉使用WebSocket上傳音頻數據並獲取識別結果。
  3. 熟悉對敏感權限的動態申請方式,本項目的敏感權限爲MICROPHONE
  4. 關於如何搭建即時語音識別服務,可以參考我的另外一篇文章:《識別準確率竟如此高,即時語音識別服務》

使用環境:

  • API 9
  • DevEco Studio 4.0 Release
  • Windows 11
  • Stage模型
  • ArkTS語言

所需權限:

  1. ohos.permission.MICROPHONE

效果圖:

核心代碼:

src/main/ets/utils/Permission.ets是動態申請權限的工具:

import bundleManager from '@ohos.bundle.bundleManager';
import abilityAccessCtrl, { Permissions } from '@ohos.abilityAccessCtrl';

async function checkAccessToken(permission: Permissions): Promise<abilityAccessCtrl.GrantStatus> {
  let atManager = abilityAccessCtrl.createAtManager();
  let grantStatus: abilityAccessCtrl.GrantStatus;

  // 獲取應用程序的accessTokenID
  let tokenId: number;
  try {
    let bundleInfo: bundleManager.BundleInfo = await bundleManager.getBundleInfoForSelf(bundleManager.BundleFlag.GET_BUNDLE_INFO_WITH_APPLICATION);
    let appInfo: bundleManager.ApplicationInfo = bundleInfo.appInfo;
    tokenId = appInfo.accessTokenId;
  } catch (err) {
    console.error(`getBundleInfoForSelf failed, code is ${err.code}, message is ${err.message}`);
  }

  // 校驗應用是否被授予權限
  try {
    grantStatus = await atManager.checkAccessToken(tokenId, permission);
  } catch (err) {
    console.error(`checkAccessToken failed, code is ${err.code}, message is ${err.message}`);
  }

  return grantStatus;
}

export async function checkPermissions(permission: Permissions): Promise<boolean> {
  let grantStatus: abilityAccessCtrl.GrantStatus = await checkAccessToken(permission);

  if (grantStatus === abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED) {
    return true
  } else {
    return false
  }
}

src/main/ets/utils/Recorder.ets是錄音工具類,進行錄音和獲取錄音數據。

import audio from '@ohos.multimedia.audio';
import { delay } from './Utils';

export default class AudioCapturer {
  private audioCapturer: audio.AudioCapturer | undefined = undefined
  private isRecording: boolean = false
  private audioStreamInfo: audio.AudioStreamInfo = {
    samplingRate: audio.AudioSamplingRate.SAMPLE_RATE_16000, // 音頻採樣率
    channels: audio.AudioChannel.CHANNEL_1, // 錄音通道數
    sampleFormat: audio.AudioSampleFormat.SAMPLE_FORMAT_S16LE,
    encodingType: audio.AudioEncodingType.ENCODING_TYPE_RAW  // 音頻編碼類型
  }
  private audioCapturerInfo: audio.AudioCapturerInfo = {
    // 音源類型,使用SOURCE_TYPE_VOICE_RECOGNITION會有減噪功能,如果設備不支持,該用普通麥克風:SOURCE_TYPE_MIC
    source: audio.SourceType.SOURCE_TYPE_VOICE_RECOGNITION,
    capturerFlags: 0 // 音頻採集器標誌
  }
  private audioCapturerOptions: audio.AudioCapturerOptions = {
    streamInfo: this.audioStreamInfo,
    capturerInfo: this.audioCapturerInfo
  }

  // 初始化,創建實例,設置監聽事件
  constructor() {
    // 創建AudioCapturer實例
    audio.createAudioCapturer(this.audioCapturerOptions, (err, capturer) => {
      if (err) {
        console.error(`創建錄音器失敗, 錯誤碼:${err.code}, 錯誤信息:${err.message}`)
        return
      }
      this.audioCapturer = capturer
      console.info('創建錄音器成功')
    });
  }

  // 開始一次音頻採集
  async start(callback: (state: number, data?: ArrayBuffer) => void) {
    // 當且僅當狀態爲STATE_PREPARED、STATE_PAUSED和STATE_STOPPED之一時才能啓動採集
    let stateGroup = [audio.AudioState.STATE_PREPARED, audio.AudioState.STATE_PAUSED, audio.AudioState.STATE_STOPPED]
    if (stateGroup.indexOf(this.audioCapturer.state) === -1) {
      console.error('啓動錄音失敗')
      callback(audio.AudioState.STATE_INVALID)
      return
    }
    // 啓動採集
    await this.audioCapturer.start()
    this.isRecording = true
    let bufferSize = 1920
    // let bufferSize: number = await this.audioCapturer.getBufferSize();

    while (this.isRecording) {
      let buffer = await this.audioCapturer.read(bufferSize, true)
      if (buffer === undefined) {
        console.error('讀取錄音數據失敗')
      } else {
        callback(audio.AudioState.STATE_RUNNING, buffer)
      }
    }
    callback(audio.AudioState.STATE_STOPPED)
  }

  // 停止採集
  async stop() {
    this.isRecording = false
    // 只有採集器狀態爲STATE_RUNNING或STATE_PAUSED的時候纔可以停止
    if (this.audioCapturer.state !== audio.AudioState.STATE_RUNNING && this.audioCapturer.state !== audio.AudioState.STATE_PAUSED) {
      console.warn('Capturer is not running or paused')
      return
    }
    await delay(200)
    // 停止採集
    await this.audioCapturer.stop()
    if (this.audioCapturer.state.valueOf() === audio.AudioState.STATE_STOPPED) {
      console.info('錄音停止')
    } else {
      console.error('錄音停止失敗')
    }
  }

  // 銷燬實例,釋放資源
  async release() {
    // 採集器狀態不是STATE_RELEASED或STATE_NEW狀態,才能release
    if (this.audioCapturer.state === audio.AudioState.STATE_RELEASED || this.audioCapturer.state === audio.AudioState.STATE_NEW) {
      return
    }
    // 釋放資源
    await this.audioCapturer.release()
  }
}

還需要一些其他的工具函數src/main/ets/utils/Utils.ets,這個主要用於睡眠等待:

// 睡眠
export function delay(milliseconds : number) {
  return new Promise(resolve => setTimeout( resolve, milliseconds));
}

還需要在src/main/module.json5添加所需要的權限,注意是在module中添加,關於字段說明,也需要在各個的string.json添加:

    "requestPermissions": [
      {
        "name": "ohos.permission.MICROPHONE",
        "reason": "$string:record_reason",
        "usedScene": {
          "abilities": [
            "EntryAbility"
          ],
          "when": "always"
        }
      }
    ]

頁面代碼如下:

import abilityAccessCtrl, { Permissions } from '@ohos.abilityAccessCtrl';
import common from '@ohos.app.ability.common';
import webSocket from '@ohos.net.webSocket';
import AudioCapturer from '../utils/Recorder';
import promptAction from '@ohos.promptAction';
import { checkPermissions } from '../utils/Permission';
import audio from '@ohos.multimedia.audio';

// 需要動態申請的權限
const permissions: Array<Permissions> = ['ohos.permission.MICROPHONE'];
// 獲取程序的上下文
const context = getContext(this) as common.UIAbilityContext;

@Entry
@Component
struct Index {
  @State recordBtnText: string = '按下錄音'
  @State speechResult: string = ''
  private offlineResult = ''
  private onlineResult = ''
  // 語音識別WebSocket地址
  private asrWebSocketUrl = "ws://192.168.0.100:10095"
  // 錄音器
  private audioCapturer?: AudioCapturer;
  // 創建WebSocket
  private ws;

  // 頁面顯示時
  async onPageShow() {
    // 判斷是否已經授權
    let promise = checkPermissions(permissions[0])
    promise.then((result) => {
      if (result) {
        // 初始化錄音器
        if (this.audioCapturer == null) {
          this.audioCapturer = new AudioCapturer()
        }
      } else {
        this.reqPermissionsAndRecord(permissions)
      }
    })
  }

  // 頁面隱藏時
  async onPageHide() {
    if (this.audioCapturer != null) {
      this.audioCapturer.release()
    }
  }

  build() {
    Row() {
      RelativeContainer() {
        Text(this.speechResult)
          .id("resultText")
          .width('95%')
          .maxLines(10)
          .fontSize(18)
          .margin({ top: 10 })
          .alignRules({
            top: { anchor: '__container__', align: VerticalAlign.Top },
            middle: { anchor: '__container__', align: HorizontalAlign.Center }
          })
        // 錄音按鈕
        Button(this.recordBtnText)
          .width('90%')
          .id("recordBtn")
          .margin({ bottom: 10 })
          .alignRules({
            bottom: { anchor: '__container__', align: VerticalAlign.Bottom },
            middle: { anchor: '__container__', align: HorizontalAlign.Center }
          })
          .onTouch((event) => {
            switch (event.type) {
              case TouchType.Down:
                console.info('按下按鈕')
              // 判斷是否有權限
                let promise = checkPermissions(permissions[0])
                promise.then((result) => {
                  if (result) {
                    // 開始錄音
                    this.startRecord()
                    this.recordBtnText = '錄音中...'
                  } else {
                    // 申請權限
                    this.reqPermissionsAndRecord(permissions)
                  }
                })
                break
              case TouchType.Up:
                console.info('鬆開按鈕')
                // 停止錄音
                this.stopRecord()
                this.recordBtnText = '按下錄音'
                break
            }
          })
      }
      .height('100%')
      .width('100%')
    }
    .height('100%')
  }

  // 開始錄音
  startRecord() {
    this.setWebSocketCallback()
    this.ws.connect(this.asrWebSocketUrl, (err) => {
      if (!err) {
        console.log("WebSocket連接成功");
        let jsonData = '{"mode": "2pass", "chunk_size": [5, 10, 5], "chunk_interval": 10, ' +
          '"wav_name": "HarmonyOS", "is_speaking": true, "itn": false}'
        // 要發完json數據才能錄音
        this.ws.send(jsonData)
        // 開始錄音
        this.audioCapturer.start((state, data) => {
          if (state == audio.AudioState.STATE_STOPPED) {
            console.info('錄音結束')
            // 錄音結束,要發消息告訴服務器,結束識別
            let jsonData = '{"is_speaking": false}'
            this.ws.send(jsonData)
          } else if (state == audio.AudioState.STATE_RUNNING) {
            // 發送語音數據
            this.ws.send(data, (err) => {
              if (err) {
                console.log("WebSocket發送數據失敗,錯誤信息:" + JSON.stringify(err))
              }
            });
          }
        })
      } else {
        console.log("WebSocket連接失敗,錯誤信息: " + JSON.stringify(err));
      }
    });
  }

  // 停止錄音
  stopRecord() {
    if (this.audioCapturer != null) {
      this.audioCapturer.stop()
    }
  }

  // 綁定WebSocket事件
  setWebSocketCallback() {
    // 創建WebSocket
    this.ws = webSocket.createWebSocket();
    // 接收WebSocket消息
    this.ws.on('message', (err, value: string) => {
      console.log("WebSocket接收消息,結果如下:" + value)
      // 解析數據
      let result = JSON.parse(value)
      let is_final = result['is_final']
      let mode = result['mode']
      let text = result['text']
      if (mode == '2pass-offline') {
        this.offlineResult = this.offlineResult + text
        this.onlineResult = ''
      } else {
        this.onlineResult = this.onlineResult + text
      }
      this.speechResult = this.offlineResult + this.onlineResult
      // 如果是最後的數據就關閉WebSocket
      if (is_final) {
        this.ws.close()
      }
    });
    // WebSocket關閉事件
    this.ws.on('close', () => {
      console.log("WebSocket關閉連接");
    });
    // WebSocket發生錯誤事件
    this.ws.on('error', (err) => {
      console.log("WebSocket出現錯誤,錯誤信息: " + JSON.stringify(err));
    });
  }

  // 申請權限
  reqPermissionsAndRecord(permissions: Array<Permissions>): void {
    let atManager = abilityAccessCtrl.createAtManager();
    // requestPermissionsFromUser會判斷權限的授權狀態來決定是否喚起彈窗
    atManager.requestPermissionsFromUser(context, permissions).then((data) => {
      let grantStatus: Array<number> = data.authResults;
      let length: number = grantStatus.length;
      for (let i = 0; i < length; i++) {
        if (grantStatus[i] === 0) {
          // 用戶授權,可以繼續訪問目標操作
          console.info('授權成功')
          if (this.audioCapturer == null) {
            this.audioCapturer = new AudioCapturer()
          }
        } else {
          promptAction.showToast({ message: '授權失敗,需要授權才能錄音' })
          return;
        }
      }
    }).catch((err) => {
      console.error(`requestPermissionsFromUser failed, code is ${err.code}, message is ${err.message}`);
    })
  }
}
小夜