Repository navigation
Fastrpc interrupted invoke v4 - #1469
Anandu Krishnan E (quic-anane) wants to merge 4 commits into
Conversation
…interrupted invoke" This reverts commit 9b45929.
…ee() fastrpc_context_alloc() takes a reference on fl via fastrpc_user_get() for the lifetime of every context, and fastrpc_context_free() always unlinks the context from fl->pending before releasing that reference with fastrpc_user_put(). This means fl->refcount cannot reach zero - and fastrpc_user_free() cannot run - while any context is still linked on fl->pending: whichever context's release happens to be the one that drops the last reference has, by construction, already removed itself from the list a few lines earlier in the very same call. The list_for_each_entry_safe() loop over fl->pending in fastrpc_user_free() can therefore never find anything to iterate. It has been unreachable since fl's own lifetime was switched from file-close-synchronous teardown to kref-counted teardown; it was never updated to reflect that the list it walks is now guaranteed empty by the time this callback can run. Remove it. Link: https://lore.kernel.org/all/20261117-interrupted-invoke-v4-v4-1-2320a8a7eda2@oss.qualcomm.com/ Fixes: e85eb5f ("misc: fastrpc: fix use-after-free of fastrpc_user in workqueue context") Signed-off-by: Anandu Krishnan E <anandu.e@oss.qualcomm.com>
…emoval fastrpc_invoke_send() takes a second reference on the invoke context before calling rpmsg_send(), to be dropped later by whichever side consumes the eventual DSP response: fastrpc_rpmsg_callback() for a real reply, via schedule_work(&ctx->put_work). fastrpc_notify_users(), called from fastrpc_rpmsg_remove() when the rpmsg channel goes away (e.g. on subsystem restart), only completes each pending context with -EPIPE so its waiter can return - it never schedules the matching put. Since the channel is gone, no response will ever arrive to trigger it through the normal callback path either. Every context that was genuinely in flight (successfully sent, not yet answered) at the time of an SSR therefore leaks that second reference, and with it the context, its DMA maps, and the reference it holds on fl and cctx. Schedule the same put_work that a real response would have triggered, exactly as fastrpc_rpmsg_callback() does. rpmsg guarantees .callback and .remove are mutually exclusive for a given device, so there is no concurrent real response that could race with this. Link: https://lore.kernel.org/all/20261117-interrupted-invoke-v4-v4-2-2320a8a7eda2@oss.qualcomm.com/ Fixes: c68cfb7 ("misc: fastrpc: Add support for context Invoke method") Cc: stable@kernel.org Signed-off-by: Anandu Krishnan E <anandu.e@oss.qualcomm.com>
…ted invoke fastrpc invokes work by sending an RPC message to the DSP and blocking in wait_for_completion_interruptible() until the DSP responds. If a signal arrives during this wait, the syscall returns -ERESTARTSYS and the invoke context, which holds the in-flight DMA buffers and completion state, is left stranded in fl->pending. On the next syscall attempt (either auto-restarted by the kernel via SA_RESTART or manually retried by user-space after EINTR), a fresh context is allocated and the RPC message is re-sent to the DSP. This has two consequences: - The original context leaks in fl->pending until the file is closed. - The DSP receives a duplicate invocation. If the DSP was mid-way through processing the first request and had issued a reverse RPC call back to the host, the retry sends a new forward request instead of the expected reverse-RPC response. The DSP thread waiting for that response is never woken, causing a hang. Fix this by saving the interrupted context to a new fl->interrupted list on -ERESTARTSYS, instead of freeing it or leaving it on fl->pending. The context keeps the same reference it already held; only list membership changes, so the refcount stays symmetric no matter how many times a context is interrupted and retried. When the same thread retries with a matching handle and sc, the context is pulled back onto fl->pending and the call jumps straight to the wait, skipping context allocation and message re-send. A mismatched handle/sc falls through to a normal fresh allocation, so a thread that issues a different call after an interrupted one is never wedged. A context on fl->interrupted can be matched and reclaimed by fastrpc_context_restore_interrupted() on a retry while a concurrent kref_put() from an unrelated drain (SSR, fd close, or a delayed DSP response) has already dropped its refcount to zero and is waiting on fl->lock to unlink it in fastrpc_context_free(). Without a check, the retry can win that lock race, hand back a context already committed to being freed, and use it after the other side frees it. kref_get_unless_zero() closes the window: it fails if the refcount is already zero, so the retry falls through to a fresh allocation instead of resurrecting a dying context. Distinguish an interruption of the send itself (rpmsg_send() returning -ERESTARTSYS before the message ever reaches the DSP) from an interruption of the wait that follows a successful send: only the latter is safe to park and resume without resending, since nothing is actually in flight in the former case. Rather than track this with an extra flag, give the two failure sites their own cleanup instead of funnelling both into the same bail: label. ctx->args is now owned by the context (copied at alloc time) and freed exactly once by fastrpc_context_free(), rather than aliasing the caller's array, which could otherwise be freed or reused by the time a parked context is eventually drained. A context parked on fl->interrupted is reachable from a resumed retry, process exit and SSR all at once. fastrpc_context_free() now removes it from whichever list it is on before releasing its resources, using list_del_init() (plain list_del() poisons the node, which would otherwise break this same check if the context is freed via a race with one of the other drains). Because the DSP response worker (fastrpc_rpmsg_callback()) and the exit/SSR/fd-close paths can all end up scheduling the same ctx->put_work for a parked context, a put_work_scheduled flag checked and set under fl->lock ensures only one of them actually queues it. fastrpc_device_release() now also drains fl->interrupted, completing any parked context with -EPIPE and dropping the invoker's reference, so a context stuck there at fd close cannot leak it. It cannot drop the worker reference unconditionally, though: the DSP may still be using that context's buffers if it hasn't answered yet, and freeing them under it would be unsafe. fastrpc_release_current_dsp_process(), already called at the top of fastrpc_device_release(), tells the DSP to tear down the whole process; if that succeeds, the DSP is known to be done with everything for this process, so the worker reference is dropped too. If it doesn't succeed, the context is left exactly where a real response or a future SSR would still find it: still linked on fl->interrupted, and fl is left on cctx->users (the unlink moves from fastrpc_device_release() into fastrpc_user_free(), which only runs once fl truly has no references left) so fastrpc_notify_users() can still walk down to it later. The same unreachable-orphan problem exists for a timed-out kernel invoke (-ETIMEDOUT, only possible when kernel=true): the bail: path used to unconditionally unlink the context from fl->pending before dropping its reference, even though the DSP may still answer later. Leave it linked on fl->pending in that case instead, so fastrpc_notify_users() can still find and release it if the channel is torn down before that answer arrives. Remove the obsolete invoke_interrupted_mmaps mechanism from fastrpc_channel_ctx; context resources are now kept alive through the context refcount rather than by migrating mmaps to a channel-level list. Link: https://lore.kernel.org/all/20261117-interrupted-invoke-v4-v4-3-2320a8a7eda2@oss.qualcomm.com/ Fixes: 387f625 ("misc: fastrpc: handle interrupted contexts") Cc: stable@kernel.org Co-developed-by: Srinivas Kandagatla <srinivas.kandagatla@oss.qualcomm.com> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@oss.qualcomm.com> Signed-off-by: Anandu Krishnan E <anandu.e@oss.qualcomm.com>
|
Merge Check Failed: No CR Numbers Found Error: No Change Request numbers were found. Please add Change Request numbers to your pull request description in the format CRs-Fixed: 12345 or link GitHub issues that are associated with Change Requests. |
3 similar comments
|
Merge Check Failed: No CR Numbers Found Error: No Change Request numbers were found. Please add Change Request numbers to your pull request description in the format CRs-Fixed: 12345 or link GitHub issues that are associated with Change Requests. |
|
Merge Check Failed: No CR Numbers Found Error: No Change Request numbers were found. Please add Change Request numbers to your pull request description in the format CRs-Fixed: 12345 or link GitHub issues that are associated with Change Requests. |
|
Merge Check Failed: No CR Numbers Found Error: No Change Request numbers were found. Please add Change Request numbers to your pull request description in the format CRs-Fixed: 12345 or link GitHub issues that are associated with Change Requests. |
|
Merge Check Failed: No Change Task Found No associated change tasks found for CR 4652959 on any of the following entities: Entities:
CR: 4652959 Please ensure the CR has a change task associated with at least one of the entities for this branch. |
Shiraz Hashim (shashim-quic)
left a comment
There was a problem hiding this comment.
Revert "FROMLIST: misc: fastrpc: fix context leak and hang on signal-interrupted invoke"
This reverts commit 9b45929.
Add reason for revert as well in commit log.
Anandu Krishnan E (@quic-anane) pls take care of this as well. |
misc: fastrpc: fix context leak and hang on signal-interrupted invoke
Ports v4 of this fix from upstream review, replacing the stale v1
version already merged into this branch (#637).
The version already on this branch corresponds to v1. Since then the
fix went through 3 more revisions upstream that address bugs found
during review:
race, and a context leak on fd close.
SSR after fd close, fixed the same bug for -ETIMEDOUT kernel
invokes, and closed a kref_get_unless_zero() race on retry.
Reverts v1 and reapplies the full v4 series rather than layering v4
on top of the already-divergent v1 content.
misc: fastrpc: remove dead context drain in fastrpc_user_free()
misc: fastrpc: drop the DSP-response reference on channel removal
misc: fastrpc: fix context leak and hang on signal-interrupted invoke
Link: https://lore.kernel.org/all/20261117-interrupted-invoke-v4-v4-0-2320a8a7eda2@oss.qualcomm.com/
CRs-Fixed: 4652959