如何将多人连接添加到没有3D资产的ARKIT应用程序中，而是使用UITEXTVIEW进行渲染？ [Swift]

Question

因此，我正在尝试从Apple自己的示例代码中添加一个多孔元素。 Link to Sample Code page There are several examples of multipeer ARKit apps but the problem这是，在我正在使用的应用程序中，粘性音符不是3D元素，而是

出于此示例应用程序的目的，粘性音符实体没有几何形状，因此没有外观。它的锚点仅提供3D位置，它是具有外观的Note Note的屏幕空间注释。要显示它，您可以定义粘附的注释。遵循RealityKit的实体组件模型，设计一个包含注释的组件，在这种情况下，这是一种视图。请参阅ScreenSpaceComponent。

我一直在尝试使用Arthat中的Multipeer应用程序的示例，使用ARKIT元素，其中3D元素存储为资产[”协作会话“示例”或使用Modelentity几何[]，但是我还没有成功地翻译仅使用屏幕空间的应用程序。

我能够在屏幕上收到它连接到对等的消息，但这是远处的。它不会在第二个手机上渲染笔记。我被所有使它起作用的尝试都被淘汰了：(

一种替代方法是忘记将笔记绑在屏幕空间上，并使用SpriteKit将其作为常规的3D空间和2D几何事物重新创建。

系统不会渲染该系统我知道有几天的时间，我已经尝试了几天，但

一直在使用2部手机来测试这件事

我

1. 添加了p.list上的信息
2. 添加了Multipeer会话文件
3. 在与Muttipeer相关的ViewController文件上添加了代码
4. 添加了Argesturesetup（）扩展文件的代码，该文件具有粘性注释的渲染信息。
5. 有效的方法：我可以在这两款手机上看到笔记，并且收到消息说同伴加入了。我不能做的就是查看 其他用户的注释，就像我在常规3D ARKIT应用中一样。它不会 渲染。

添加到insertnewsticky函数中的内容

func insertNewSticky(_ sender: UITapGestureRecognizer)

这是我从另一个示例之一中

let anchor = ARAnchor(name: "Anchor for object placement", transform: raycastResult.worldTransform)
         arView.session.add(anchor: anchor)

：以下是手势识别器设置的完整代码

import UIKit

导入arkit

Extension viewController {

// MARK: - Gesture recognizer setup
// - Tag: AddViewTapGesture
func arViewGestureSetup() {
    let tapGesture = UITapGestureRecognizer(target: self, action: #selector(tappedOnARView))
    arView.addGestureRecognizer(tapGesture)
    
    let swipeGesture = UISwipeGestureRecognizer(target: self, action: #selector(swipedDownOnARView))
    swipeGesture.direction = .down
    arView.addGestureRecognizer(swipeGesture)
    
 

}

func stickyNoteGestureSetup(_ note: StickyNoteEntity) {
    let panGesture = UIPanGestureRecognizer(target: self, action: #selector(panOnStickyView))
    note.view?.addGestureRecognizer(panGesture)
    
    let tapOnStickyView = UITapGestureRecognizer(target: self, action: #selector(tappedOnStickyView(_:)))
    note.view?.addGestureRecognizer(tapOnStickyView)
}

// MARK: - Gesture recognizer callbacks

/// Tap gesture input handler.
/// - Tag: TapHandler
@objc
func tappedOnARView(_ sender: UITapGestureRecognizer) {
    
    // Ignore the tap if the user is editing a sticky note.
    for note in stickyNotes where note.isEditing { return }
    
    // Create a new sticky note at the tap location.
    insertNewSticky(sender)
    
    
}

/**
Hit test the feature point cloud and use any hit as the position of a new StickyNote. Otherwise, display a tip.
 - Tag: ScreenSpaceViewInsertionTag
 */
 func insertNewSticky(_ sender: UITapGestureRecognizer) {

    // Get the user's tap screen location.
    let touchLocation = sender.location(in: arView)
    
    // Cast a ray to check for its intersection with any planes.
     
     
    guard let raycastResult = arView.raycast(from: touchLocation, allowing: .estimatedPlane, alignment: .any).first 
            
            
          
     
     else {
        messageLabel.displayMessage("No surface detected, try getting closer.", duration: 2.0)
        return
   
      
    
    }

     
   

    // Create a new sticky note positioned at the hit test result's world position.
    let frame = CGRect(origin: touchLocation, size: CGSize(width: 200, height: 200))

    let note = StickyNoteEntity(frame: frame, worldTransform: raycastResult.worldTransform)
    
    // Center the sticky note's view on the tap's screen location.
    note.setPositionCenter(touchLocation)
     
 

    // Add the sticky note to the scene's entity hierarchy.
    arView.scene.addAnchor(note)

    // Add the sticky note's view to the view hierarchy.
    guard let stickyView = note.view else { return }
    arView.insertSubview(stickyView, belowSubview: trashZone)
    
    // Enable gestures on the sticky note.
    stickyNoteGestureSetup(note)

    // Save a reference to the sticky note.
    stickyNotes.append(note)
    
    // Volunteer to handle text view callbacks.
    stickyView.textView.delegate = self
     
     let anchor = ARAnchor(name: "Anchor for object placement", transform: raycastResult.worldTransform)
     arView.session.add(anchor: anchor)
    
    
    
}


/// Dismisses the keyboard.
@objc
func swipedDownOnARView(_ sender: UISwipeGestureRecognizer) {
    dismissKeyboard()
}

fileprivate func dismissKeyboard() {
    for note in stickyNotes {
        guard let textView = note.view?.textView else { continue }
        if textView.isFirstResponder {
            textView.resignFirstResponder()
            return
        }
    }
}

@objc
func tappedOnStickyView(_ sender: UITapGestureRecognizer) {
    guard let stickyView = sender.view as? StickyNoteView else { return }
    stickyView.textView.becomeFirstResponder()
}
//- Tag: PanOnStickyView
fileprivate func panStickyNote(_ sender: UIPanGestureRecognizer, _ stickyView: StickyNoteView, _ panLocation: CGPoint) {
    messageLabel.isHidden = true
    
    let feedbackGenerator = UIImpactFeedbackGenerator()
    
    switch sender.state {
    case .began:
        // Prepare the taptic engine to reduce latency in delivering feedback.
        feedbackGenerator.prepare()
        
        // Drag if the gesture is beginning.
        stickyView.stickyNote.isDragging = true
        
        // Save offsets to implement smooth panning.
        guard let frame = sender.view?.frame else { return }
        stickyView.xOffset = panLocation.x - frame.origin.x
        stickyView.yOffset = panLocation.y - frame.origin.y
        
        // Fade in the widget that's used to delete sticky notes.
        trashZone.fadeIn(duration: 0.4)
    case .ended:
        // Stop dragging if the gesture is ending.
        stickyView.stickyNote.isDragging = false
        
        // Delete the sticky note if the gesture ended on the trash widget.
        if stickyView.isInTrashZone {
            deleteStickyNote(stickyView.stickyNote)
            // ...
        } else {
            attemptRepositioning(stickyView)
        }
        
        // Fades out the widget that's used to delete sticky notes when there are no sticky notes currently being dragged.
        if !stickyNotes.contains(where: { $0.isDragging }) {
            trashZone.fadeOut(duration: 0.2)
        }
    default:
        // Update the sticky note's screen position based on the pan location, and initial offset.
        stickyView.frame.origin.x = panLocation.x - stickyView.xOffset
        stickyView.frame.origin.y = panLocation.y - stickyView.yOffset
        
        // Give feedback whenever the pan location is near the widget used to delete sticky notes.
        trashZoneThresholdFeedback(sender, feedbackGenerator)
    }
}

/// Sticky note pan-gesture handler.
/// - Tag: PanHandler
@objc
func panOnStickyView(_ sender: UIPanGestureRecognizer) {
    
    guard let stickyView = sender.view as? StickyNoteView else { return }
    
    let panLocation = sender.location(in: arView)
    
    // Ignore the pan if any StickyViews are being edited.
    for note in stickyNotes where note.isEditing { return }
    
    panStickyNote(sender, stickyView, panLocation)
}

func deleteStickyNote(_ note: StickyNoteEntity) {
    guard let index = stickyNotes.firstIndex(of: note) else { return }
    note.removeFromParent()
    stickyNotes.remove(at: index)
    note.view?.removeFromSuperview()
    note.view?.isInTrashZone = false
}

/// - Tag: AttemptRepositioning
fileprivate func attemptRepositioning(_ stickyView: StickyNoteView) {
    // Conducts a ray-cast for feature points using the panned position of the StickyNoteView
    let point = CGPoint(x: stickyView.frame.midX, y: stickyView.frame.midY)
    if let result = arView.raycast(from: point, allowing: .estimatedPlane, alignment: .any).first {
        stickyView.stickyNote.transform.matrix = result.worldTransform
    } else {
        messageLabel.displayMessage("No surface detected, unable to reposition note.", duration: 2.0)
        stickyView.stickyNote.shouldAnimate = true
    }
}

fileprivate func trashZoneThresholdFeedback(_ sender: UIPanGestureRecognizer, _ feedbackGenerator: UIImpactFeedbackGenerator) {
    
    guard let stickyView = sender.view as? StickyNoteView else { return }
    
    let panLocation = sender.location(in: trashZone)
    
    if trashZone.frame.contains(panLocation), !stickyView.isInTrashZone {
        stickyView.isInTrashZone = true
        feedbackGenerator.impactOccurred()
        
    } else if !trashZone.frame.contains(panLocation), stickyView.isInTrashZone {
        stickyView.isInTrashZone = false
        feedbackGenerator.impactOccurred()
        
    }
}

@objc
func tappedReset(_ sender: UIButton) {
    reset()
}

}

，这是ViewController文件的完整代码，

/*

请参见此示例的许可文件夹信息。

抽象的： AR体验的主视图控制器。 */

导入Uikit 导入RealityKit 进口组合 导入arkit 导入MultipErConnectivity

类ViewController：UiviewController，ArsessionDelegate {

// MARK: - Class variable declarations

@IBOutlet var arView: ARView!
@IBOutlet weak var messageLabel: MessageLabel!
var trashZone: GradientView!
var shadeView: UIView!
var resetButton: UIButton!

var keyboardHeight: CGFloat!

var stickyNotes = [StickyNoteEntity]()

var subscription: Cancellable!

//added Sat May 28 5:12pm
    var multipeerSession: MultipeerSession?

// end of added Sat May 28 5:12pm

//added Sat May 28 5:12pm
   // A dictionary to map MultiPeer IDs to ARSession ID's.
    // This is useful for keeping track of which peer created which ARAnchors.
    var peerSessionIDs = [MCPeerID: String]()
    
    var sessionIDObservation: NSKeyValueObservation?
    
    var configuration: ARWorldTrackingConfiguration?




// end of added Sat May 28 5:12pm



// MARK: - View Controller Life Cycle

override func viewDidLoad() {
    super.viewDidLoad()
    
    subscription = arView.scene.subscribe(to: SceneEvents.Update.self) { [unowned self] in
        self.updateScene(on: $0)
    }
    
    arViewGestureSetup()
    overlayUISetup()
    
    arView.session.delegate = self
}

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    
    // Add observer to the keyboardWillShowNotification to get the height of the keyboard every time it is shown
    let notificationName = UIResponder.keyboardWillShowNotification
    let selector = #selector(keyboardIsPoppingUp(notification:))
    NotificationCenter.default.addObserver(self, selector: selector, name: notificationName, object: nil)
    
}

override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    
    arView.session.delegate = self

    
    // Prevent the screen from being dimmed to avoid interuppting the AR experience.
    UIApplication.shared.isIdleTimerDisabled = true
    
    // Turn off ARView's automatically-configured session
    // to create and set up your own configuration.
    arView.automaticallyConfigureSession = false
    
    configuration = ARWorldTrackingConfiguration()

    // Enable a collaborative session.
    configuration?.isCollaborationEnabled = true
    
    // Enable realistic reflections.
    configuration?.environmentTexturing = .automatic

    // Begin the session.
    arView.session.run(configuration!)
    
    
    // Use key-value observation to monitor your ARSession's identifier.
    sessionIDObservation = observe(\.arView.session.identifier, options: [.new]) { object, change in
        print("SessionID changed to: \(change.newValue!)")
        // Tell all other peers about your ARSession's changed ID, so
        // that they can keep track of which ARAnchors are yours.
        guard let multipeerSession = self.multipeerSession else { return }
        self.sendARSessionIDTo(peers: multipeerSession.connectedPeers)
    }
    
    // Start looking for other players via MultiPeerConnectivity.
    multipeerSession = MultipeerSession(receivedDataHandler: receivedData, peerJoinedHandler:
                                        peerJoined, peerLeftHandler: peerLeft, peerDiscoveredHandler: peerDiscovered)
    
    //arView.addGestureRecognizer(UITapGestureRecognizer(target: self, action: #selector(handleTap(recognizer:))))
    
    messageLabel.displayMessage("Tap the screen to place cubes.\nInvite others to launch this app to join you.", duration: 60.0)
    
}




//peerDiscovered




func peerDiscovered(_ peer: MCPeerID) -> Bool {
      guard let multipeerSession = multipeerSession else { return false }
      
      if multipeerSession.connectedPeers.count > 3 {
          // Do not accept more than four users in the experience.
          messageLabel.displayMessage("A fifth peer wants to join the experience.\nThis app is limited to four users.", duration: 6.0)
          return false
      } else {
          return true
      }
  }
// end of added Sat May 28 5:12pm


  /// - Tag: PeerJoined
// added Sat May 28 5:12pm
 func peerJoined(_ peer: MCPeerID) {
     messageLabel.displayMessage("""
         A peer has joined the experience.
         Hold the phones next to each other.
         """, duration: 6.0)
     // Provide your session ID to the new user so they can keep track of your anchors.
     sendARSessionIDTo(peers: [peer])
 }



// end of added Sat May 28 5:12pm

// added Sat May 28 5:12pm
func peerLeft(_ peer: MCPeerID) {
    messageLabel.displayMessage("A peer has left the shared experience.")
    
    // Remove all ARAnchors associated with the peer that just left the experience.
    if let sessionID = peerSessionIDs[peer] {
        removeAllAnchorsOriginatingFromARSessionWithID(sessionID)
        peerSessionIDs.removeValue(forKey: peer)
    }
}



// end of added Sat May 28 5:12pm




//added Sat May 28 5:12pm


func receivedData(_ data: Data, from peer: MCPeerID) {
    if let collaborationData = try? NSKeyedUnarchiver.unarchivedObject(ofClass: ARSession.CollaborationData.self, from: data) {
        arView.session.update(with: collaborationData)
        return
    }
    // ...
    let sessionIDCommandString = "SessionID:"
    if let commandString = String(data: data, encoding: .utf8), commandString.starts(with: sessionIDCommandString) {
        let newSessionID = String(commandString[commandString.index(commandString.startIndex,
                                                                 offsetBy: sessionIDCommandString.count)...])
        // If this peer was using a different session ID before, remove all its associated anchors.
        // This will remove the old participant anchor and its geometry from the scene.
        if let oldSessionID = peerSessionIDs[peer] {
            removeAllAnchorsOriginatingFromARSessionWithID(oldSessionID)
        }
        
        peerSessionIDs[peer] = newSessionID
    }
}
// end of added Sat May 28 5:12pm

func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
   
            messageLabel.displayMessage("Established joint experience with a peer.")
            // ...
       
    }



func updateScene(on event: SceneEvents.Update) {
    let notesToUpdate = stickyNotes.compactMap { !$0.isEditing && !$0.isDragging ? $0 : nil }
    for note in notesToUpdate {
        // Gets the 2D screen point of the 3D world point.
        guard let projectedPoint = arView.project(note.position) else { return }
        
        // Calculates whether the note can be currently visible by the camera.
        let cameraForward = arView.cameraTransform.matrix.columns.2.xyz
        let cameraToWorldPointDirection = normalize(note.transform.translation - arView.cameraTransform.translation)
        let dotProduct = dot(cameraForward, cameraToWorldPointDirection)
        let isVisible = dotProduct < 0

        // Updates the screen position of the note based on its visibility
        note.projection = Projection(projectedPoint: projectedPoint, isVisible: isVisible)
        note.updateScreenPosition()
    }
}

func reset() {
    guard let configuration = arView.session.configuration else { return }
    arView.session.run(configuration, options: .removeExistingAnchors)
    for note in stickyNotes {
        deleteStickyNote(note)
    }
}


func session(_ session: ARSession, didFailWithError error: Error) {
    guard error is ARError else { return }
    
    let errorWithInfo = error as NSError
    let messages = [
        errorWithInfo.localizedDescription,
        errorWithInfo.localizedFailureReason,
        errorWithInfo.localizedRecoverySuggestion
    ]
    let errorMessage = messages.compactMap({ $0 }).joined(separator: "\n")
    
    DispatchQueue.main.async {
        // Present an alert informing about the error that has occurred.
        let alertController = UIAlertController(title: "The AR session failed.", message: errorMessage, preferredStyle: .alert)
        let restartAction = UIAlertAction(title: "Restart Session", style: .default) { _ in
            alertController.dismiss(animated: true, completion: nil)
            self.reset()
        }
        alertController.addAction(restartAction)
        self.present(alertController, animated: true, completion: nil)
    }
}

override var prefersStatusBarHidden: Bool {
    return true
}

override var prefersHomeIndicatorAutoHidden: Bool {
    return true
}

private func sendARSessionIDTo(peers: [MCPeerID]) {
    guard let multipeerSession = multipeerSession else { return }
    let idString = arView.session.identifier.uuidString
    let command = "SessionID:" + idString
    if let commandData = command.data(using: .utf8) {
        multipeerSession.sendToPeers(commandData, reliably: true, peers: peers)
    }
}

private func removeAllAnchorsOriginatingFromARSessionWithID(_ identifier: String) {
    guard let frame = arView.session.currentFrame else { return }
    for anchor in frame.anchors {
        guard let anchorSessionID = anchor.sessionIdentifier else { continue }
        if anchorSessionID.uuidString == identifier {
            arView.session.remove(anchor: anchor)
        }
    }
}

}

Answer 1

更新：我与Apple的Reality Kit团队的一名员工工程师进行了交谈，他向我解释说，我要完成的目标是不可行的，因为该音符具有嵌入式子类，该子类并非``可编码'' /developer.apple.com/documentation/swift/codable“ rel =“ nofollow noreferrer”>代码协议
我将不得不与与我一起工作的示例进行不同的重建注释，以确保它适合代码协议，这将确保数据可以通过