{"abstract":[{"text":"A central challenge in building robotic prostheses is the creation of a sensor-based system able to read physiological signals from the lower limb and instruct a robotic hand to perform various tasks. Existing systems typically perform discrete gestures such as pointing or grasping, by employing electromyography (EMG) or ultrasound (US) technologies to analyze muscle states. While estimating finger gestures has been done in the past by detecting prominent gestures, we are interested in detection, or inference, done in the context of fine motions that evolve over time. Examples include motions occurring when performing fine and dexterous tasks such as keyboard typing or piano playing. We consider this task as an important step towards higher adoption rates of robotic prostheses among arm amputees, as it has the potential to dramatically increase functionality in performing daily tasks. To this end, we present an end-to-end robotic system, which can successfully infer fine finger motions. This is achieved by modeling the hand as a robotic manipulator and using it as an intermediate representation to encode muscles' dynamics from a sequence of US images. We evaluated our method by collecting data from a group of subjects and demonstrating how it can be used to replay music played or text typed. To the best of our knowledge, this is the first study demonstrating these downstream tasks within an end-to-end system.","lang":"eng"}],"external_id":{"arxiv":["2202.05204"]},"title":"Towards predicting fine finger motions from ultrasound images via kinematic representation","_id":"18217","date_updated":"2024-10-09T12:00:32Z","article_processing_charge":"No","month":"07","date_published":"2023-07-04T00:00:00Z","status":"public","publication_status":"published","citation":{"ista":"Zadok D, Salzman O, Wolf A, Bronstein AM. 2023. Towards predicting fine finger motions from ultrasound images via kinematic representation. 2023 IEEE International Conference on Robotics and Automation. ICRA: Conference on Robotics and Automation vol. 27.","ieee":"D. Zadok, O. Salzman, A. Wolf, and A. M. Bronstein, “Towards predicting fine finger motions from ultrasound images via kinematic representation,” in <i>2023 IEEE International Conference on Robotics and Automation</i>, London, United Kingdom, 2023, vol. 27.","apa":"Zadok, D., Salzman, O., Wolf, A., &#38; Bronstein, A. M. (2023). Towards predicting fine finger motions from ultrasound images via kinematic representation. In <i>2023 IEEE International Conference on Robotics and Automation</i> (Vol. 27). London, United Kingdom: IEEE. <a href=\"https://doi.org/10.1109/icra48891.2023.10160601\">https://doi.org/10.1109/icra48891.2023.10160601</a>","mla":"Zadok, Dean, et al. “Towards Predicting Fine Finger Motions from Ultrasound Images via Kinematic Representation.” <i>2023 IEEE International Conference on Robotics and Automation</i>, vol. 27, IEEE, 2023, doi:<a href=\"https://doi.org/10.1109/icra48891.2023.10160601\">10.1109/icra48891.2023.10160601</a>.","ama":"Zadok D, Salzman O, Wolf A, Bronstein AM. Towards predicting fine finger motions from ultrasound images via kinematic representation. In: <i>2023 IEEE International Conference on Robotics and Automation</i>. Vol 27. IEEE; 2023. doi:<a href=\"https://doi.org/10.1109/icra48891.2023.10160601\">10.1109/icra48891.2023.10160601</a>","short":"D. Zadok, O. Salzman, A. Wolf, A.M. Bronstein, in:, 2023 IEEE International Conference on Robotics and Automation, IEEE, 2023.","chicago":"Zadok, Dean, Oren Salzman, Alon Wolf, and Alex M. Bronstein. “Towards Predicting Fine Finger Motions from Ultrasound Images via Kinematic Representation.” In <i>2023 IEEE International Conference on Robotics and Automation</i>, Vol. 27. IEEE, 2023. <a href=\"https://doi.org/10.1109/icra48891.2023.10160601\">https://doi.org/10.1109/icra48891.2023.10160601</a>."},"language":[{"iso":"eng"}],"author":[{"first_name":"Dean","last_name":"Zadok","full_name":"Zadok, Dean"},{"first_name":"Oren","last_name":"Salzman","full_name":"Salzman, Oren"},{"full_name":"Wolf, Alon","first_name":"Alon","last_name":"Wolf"},{"last_name":"Bronstein","orcid":"0000-0001-9699-8730","first_name":"Alexander","id":"58f3726e-7cba-11ef-ad8b-e6e8cb3904e6","full_name":"Bronstein, Alexander"}],"year":"2023","scopus_import":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","doi":"10.1109/icra48891.2023.10160601","oa_version":"Preprint","publication":"2023 IEEE International Conference on Robotics and Automation","day":"04","publisher":"IEEE","main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2202.05204"}],"quality_controlled":"1","publication_identifier":{"eisbn":["9798350323658"]},"conference":{"start_date":"2023-05-29","end_date":"2023-06-02","location":"London, United Kingdom","name":"ICRA: Conference on Robotics and Automation"},"volume":27,"oa":1,"type":"conference","intvolume":"        27","date_created":"2024-10-08T12:50:55Z","extern":"1"}